LaTeXML::Core::Token(3pm)

Exported functions

"$catcode = CC_ESCAPE;"

Constants for the category codes:

  CC_BEGIN, CC_END, CC_MATH, CC_ALIGN, CC_EOL,
  CC_PARAM, CC_SUPER, CC_SUB, CC_IGNORE,
  CC_SPACE, CC_LETTER, CC_OTHER, CC_ACTIVE,
  CC_COMMENT, CC_INVALID, CC_CS.

[The last 2 are (apparent) extensions, with catcodes 16 and 17, respectively].

"$token = Token($string,$cc);"

Creates a LaTeXML::Core::Token with the given content and catcode. The following shorthand versions are also exported for convenience:

  T_BEGIN, T_END, T_MATH, T_ALIGN, T_PARAM,
  T_SUB, T_SUPER, T_SPACE, T_LETTER($letter),
  T_OTHER($char), T_ACTIVE($char),
  T_COMMENT($comment), T_CS($cs)

"@tokens = Explode($string);"

Returns a list of the tokens corresponding to the characters in $string. All tokens have catcode CC_OTHER, except for spaces which have catcode CC_SPACE.

"@tokens = ExplodeText($string);"

Returns a list of the tokens corresponding to the characters in $string. All (roman) letters have catcode CC_LETTER, all others have catcode CC_OTHER, except for spaces which have catcode CC_SPACE.

"UnTeX($object, $suppress_linebreaks);"

Converts $object to a string containing TeX that created it (or could have). Note that this is not necessarily the original TeX code; expansions or other substitutions may have taken place.

Line-breaking of the generated TeX can be explicitly requested or disabled by passing 0 or 1 as the second $suppress_linebreaks argument. The default behavior of line-breaking is controlled by the global State value "SUPPRESS_UNTEX_LINEBREAKS".

Methods

"@tokens = $object->unlist;"

Return a list of the tokens making up this $object.

"$string = $object->toString;"

Return a string representing $object.

"$string = $token->getCSName;"

Return the string or character part of the $token; for the special category codes, returns the standard string (eg. "T_BEGIN->getCSName" returns "{").

"$string = $token->getString;"

Return the string or character part of the $token.

"$code = $token->getCharcode;"

Return the character code of the character part of the $token, or 256 if it is a control sequence.

"$code = $token->getCatcode;"

Return the catcode of the $token.