---
name: "CQL::Lexer"
section: "3pm"
platform: "linux"
package: "libcql-parser-perl"
version: "1.13-2"
release: "Debian 13 (Trixie)"
platforms: ["linux"]
url: "https://new.carta.tech/CQL::Lexer"
---

# CQL::Lexer(3pm)

a lexical analyzer for CQL

## NAME

CQL::Lexer - a lexical analyzer for CQL

## SYNOPSIS

```
    my $lexer = CQL::Lexer->new();
    $lexer->tokenize( 'foo and bar' );
    my @tokens = $lexer->getTokens();
```

## DESCRIPTION

CQL::Lexer is lexical analyzer for a string of CQL. Once you've got a CQL::Lexer object you can tokenize a CQL string into CQL::Token objects. Ordinarily you'll never want to do this yourself since CQL::Parser calls CQL::Lexer for you.

CQL::Lexer uses Stevan Little's lovely String::Tokenizer in the background, and does a bit of analysis afterwards to handle some peculiarities of  CQL: double quoted strings, \<, \<=, etc.

## METHODS

### new()

The constructor.

### tokenize()

Pass in a string of CQL to tokenize. This initializes the lexer with  data so that you can retrieve tokens.

### getTokens()

Returns a list of all the tokens.

### token()

Returns the current token.

### nextToken()

Returns the next token, or undef if there are more tokens to retrieve from the lexer.

### prevToken()

Returns the previous token, or undef if there are no tokens prior to the current token.

### reset()

Resets the iterator to start reading tokens from the beginning.