---
name: "xtokid"
section: "1"
platform: "linux"
package: "id-utils"
version: "4.6.28-20200521ss15dab+b1"
release: "Debian 12 (Bookworm)"
platforms: ["linux"]
url: "https://new.carta.tech/xtokid"
---

# xtokid(1)

Print all tokens found in a source file

## NAME

xtokid - Print all tokens found in a source file

## SYNOPSIS

**xtokid** \[*OPTION*\]... \[*FILE*\]...

## DESCRIPTION

Print all tokens found in a source file.

- **-i**, **--include**=*LANGS*

  include languages in LANGS (default: "C C++ asm")

- **-x**, **--exclude**=*LANGS*

  exclude languages in LANGS

**-l**, **--lang-option**=*L*:OPT pass OPT as a default for language L (see below)

- **-m**, **--lang-map**=*MAPFILE*

  use MAPFILE to map file names onto source language

- **-d**, **--default-lang**=*LANG*

  make LANG the default source language

- **-p**, **--prune**=*NAMES*

  exclude the named files and/or directories

- **--files0-from**=*F*

  tokenize only the files specified by NUL-terminated names in file F

- **--help**

  display this help and exit **--version**           output version information and exit

The following arguments apply to the language-specific scanners:

### C language:

- **-k**,--keep=CHARS

  Allow CHARS in single-token strings, keep the result

- **-i**,--ignore=CHARS

  Allow CHARS in single-token strings, toss the result

- **-u**,--strip-underscore

  Strip a leading underscore from single-token strings

### C++ language:

- **-k**,--keep=CHARS

  Allow CHARS in single-token strings, keep the result

- **-i**,--ignore=CHARS

  Allow CHARS in single-token strings, toss the result

- **-u**,--strip-underscore

  Strip a leading underscore from single-token strings

### Java language:

- **-k**,--keep=CHARS

  Allow CHARS in single-token strings, keep the result

- **-i**,--ignore=CHARS

  Allow CHARS in single-token strings, toss the result

- **-u**,--strip-underscore

  Strip a leading underscore from single-token strings

### Assembly language:

- **-c**,--comment=CHARS

  Any of CHARS starts a comment until end-of-line

- **-k**,--keep=CHARS

  Allow CHARS in tokens, and keep the result

- **-i**,--ignore=CHARS

  Allow CHARS in tokens, and toss the result

- **-u**,--strip-underscore

  Strip a leading underscore from tokens

- **-n**,--no-cpp

  Don't handle C pre-processor directives

### Text language:

- **-i**,--include=CHAR-CLASS

  Treat characters of CHAR-CLASS as token constituents

- **-x**,--exclude=CHAR-CLASS

  Treat characters of CHAR-CLASS as token delimiters

### Perl language:

- **-i**,--include=CHAR-CLASS

  Treat characters of CHAR-CLASS as token constituents

- **-x**,--exclude=CHAR-CLASS

  Treat characters of CHAR-CLASS as token delimiters

- **-d**,--dtags

  Include documentation tags

Lisp language:

## REPORTING BUGS

Report bugs to bug-idutils@gnu.org

## SEE ALSO

The full documentation for the **id-utils** package is maintained as a Texinfo manual.  If the **info** and **id-utils** packages are properly installed, the command

-

  **info 'ID database'**

should give you access to the complete manual.