next up previous contents
Next: A..2 EBNF for preprocessor Up: A. CM description file Previous: A. CM description file   Contents

A..1 Lexical Analysis

The CM parser employs a context-sensitive scanner. In many cases this avoids the need for ``escape characters'' or other lexical devices that would make writing description files cumbersome. (The downside of this is that it increases the complexity of both documentation and implementation.)

The scanner skips all nestable SML-style comments (enclosed with (* and *)).

Lines starting with #line may list up to three fields separated by white space. The first field is taken as a line number and the last field (if more than one field is present) as a file name. The optional third (middle) field specifies a column number. A line of this form resets the scanner's idea about the name of the file that it is currently processing and about the current position within that file. If no file is specified, the default is the current file. If no column is specified, the default is the first column of the (specified) line. This feature is meant for program-generators or tools such as noweb but is not intended for direct use by programmers.

The following lexical classes are recognized:

Namespace specifiers:
structure, signature, functor, or funsig. These keywords are recognized everywhere.
CM keywords:
group, Group, GROUP, library, Library, LIBRARY, is, IS. These keywords are recognized everywhere except within ``preprocessor'' lines (lines starting with #) or following one of the namespace specifiers.
Preprocessor control keywords:
#if, #elif, #else, #endif, #error. These keywords are recognized only at the beginning of the line and indicate the start of a ``preprocessor'' line. The initial # character may be separated from the rest of the token by white space (but not by comments).
Preprocessor operator keywords:
defined, div, mod, andalso, orelse, not. These keywords are recognized only when they occur within ``preprocessor'' lines. Even within such lines, they are not recognized as keywords when they directly follow a namespace specifier--in which case they are considered SML identifiers.
SML identifiers ( mlid):
Recognized SML identifiers include all legal identifiers as defined by the SML language definition. (CM also recognizes some tokens as SML identifiers that are really keywords according to the SML language definition. However, this can never cause problems in practice.) SML identifiers are recognized only when they directly follow one of the namespace specifiers.
CM identifiers ( cmid):
CM identifiers have the same form as those ML identifiers that are made up solely of letters, decimal digits, apostrophes, and underscores. CM identifiers are recognized when they occur within ``preprocessor'' lines, but not when they directly follow some namespace specifier.
Numbers ( number):
Numbers are non-empty sequences of decimal digits. Numbers are recognized only within ``preprocessor'' lines.
Preprocessor operators:
The following unary and binary operators are recognized when they occur within ``preprocessor'' lines: +, -, *, /, %, <>, !=, <=, <, >=, >, ==, =, $\tilde{~}$, &&, ||, !. Of these, the following (``C-style'') operators are considered obsolete and trigger a warning message16 as long as CM.Control.warn_obsolete is set to true: /, %, !=, ==, &&, ||, !.
Standard path names ( stdpn):
Any non-empty sequence of upper- and lower-case letters, decimal digits, and characters drawn from '_.;,!%&$+/<=>?@$\tilde{~}$|#*-^ that occurs outside of ``preprocessor'' lines and is neither a namespace specifier nor a CM keyword will be recognized as a stardard path name. Strings that lexically constitute standard path names are usually--but not always--interpreted as file names. Sometimes they are simply taken as literal strings. When they act as file names, they will be interpreted according to CM's standard syntax (see Section 3.2). (Member class names, names of privileges, and many tool optios are also specified as standard path names even though in these cases no actual file is being named.)
Native path names ( ntvpn):
A token that has the form of an SML string is considered a native path name. The same rules as in SML regarding escape characters apply. Like their ``standard'' counterparts, native path names are not always used to actually name files, but when they are, they use the native file name syntax of the underlying operating system.
Punctuation:
A colon : is recognized as a token everywhere except within ``preprocessor'' lines. Parentheses () are recognized everywhere.


next up previous contents
Next: A..2 EBNF for preprocessor Up: A. CM description file Previous: A. CM description file   Contents
Matthias Blume
2001-07-19