Automatic generation of efficient lexical processors using finite state techniques

Walter L. Johnson, James H. Porter, Stephanie I. Ackley, Douglas T. Ross

Communications of the ACM · 1968 · 68 citations · 8 references

DOIFull text

Open access

Concepts

Abstract

The practical application of the theory of finite-state automata to automatically generate lexical processors is dealt with in this tutorial article by the use of the AED RWORD system, developed at M.I.T. as part of the AED-1 system. This system accepts as input descriptions of the multicharacter items or of words allowable in a language given in terms of a subset of regular expressions. The output of the system is a lexical processor which reads a string of characters and combines them into the items as defined by the regular expressions. Each output item is identified by a code number together with a pointer to a block of storage containing the characters and character count in the item.

References

8