---
name: "Genlex"
section: "3o"
platform: "linux"
package: "ocaml-man"
version: "4.13.1-4"
release: "Debian 12 (Bookworm)"
platforms: ["linux"]
url: "https://new.carta.tech/Genlex"
---

# Genlex(3o)

A generic lexical analyzer.

## NAME

Genlex - A generic lexical analyzer.

## Module

Module   Genlex

## Documentation

Module **Genlex**\
  :  **sig end**

A generic lexical analyzer.

This module implements a simple 'standard' lexical analyzer, presented as a function from character streams to token streams. It implements roughly the lexical conventions of OCaml, but is parameterized by the set of keywords of your language.

Example: a lexer suitable for a desk calculator is obtained by

```
let lexer = make_lexer ["+"; "-"; "*"; "/"; "let"; "="; "("; ")"]
```

The associated parser would be a function from  **token stream** to, for instance,  **int** , and would have rules such as:

```
     let rec parse_expr = parser
       | [< n1 = parse_atom; n2 = parse_remainder n1 >] -> n2
     and parse_atom = parser
       | [< 'Int n >] -> n
       | [< 'Kwd "("; n = parse_expr; 'Kwd ")" >] -> n
     and parse_remainder n1 = parser
       | [< 'Kwd "+"; n2 = parse_expr >] -> n1 + n2
       | [< >] -> n1

```

One should notice that the use of the  **parser** keyword and associated notation for streams are only available through camlp4 extensions. This means that one has to preprocess its sources e. g. by using the **"-pp"** command-line switch of the compilers.

*type token* =\
  \| Kwd **of** **string**\
  \| Ident **of** **string**\
  \| Int **of** **int**\
  \| Float **of** **float**\
  \| String **of** **string**\
  \| Char **of** **char**

The type of tokens. The lexical classes are:  **Int** and  **Float** for integer and floating-point numbers;  **String** for string literals, enclosed in double quotes;  **Char** for character literals, enclosed in single quotes;  **Ident** for identifiers (either sequences of letters, digits, underscores and quotes, or sequences of 'operator characters' such as **+** ,  **\*** , etc); and  **Kwd** for keywords (either identifiers or single 'special characters' such as  **(** ,  **}** , etc).

*val make\_lexer* :  **string list -\> char Stream.t -\> token Stream.t**

Construct the lexer function. The first argument is the list of keywords. An identifier  **s** is returned as  **Kwd s** if  **s** belongs to this list, and as  **Ident s** otherwise. A special character  **s** is returned as  **Kwd s** if  **s** belongs to this list, and cause a lexical error (exception **Stream.Error** with the offending lexeme as its parameter) otherwise. Blanks and newlines are skipped. Comments delimited by  **(\*** and  **\*)** are skipped as well, and can be nested. A  **Stream.Failure** exception is raised if end of stream is unexpectedly reached.