--- name: "tok" section: "3" platform: "macos" package: "ksh" version: "42" release: "macOS 26.5" platforms: ["macos"] url: "https://new.carta.tech/tok" --- # tok(3) space separated token stream routines ## NAME tok - space separated token stream routines ## SYNOPSIS $1$2#include \#include \#include \#include \#include \#include \#include \#include \ $1$2void\* tokopen(char\* string)void\* tokopen(char\* string)void\* tokopen(char\* string)void\* tokopen(char\* string)void\* tokopen(char\* string)void\* tokopen(char\* string)void\* tokopen(char\* string)void\* tokopen(char\* string) $1$2char\* tokread(void\* tok)char\* tokread(void\* tok)char\* tokread(void\* tok)char\* tokread(void\* tok)char\* tokread(void\* tok)char\* tokread(void\* tok)char\* tokread(void\* tok)char\* tokread(void\* tok) $1$2void tokclose(void\* tok)void tokclose(void\* tok)void tokclose(void\* tok)void tokclose(void\* tok)void tokclose(void\* tok)void tokclose(void\* tok)void tokclose(void\* tok)void tokclose(void\* tok) $1$2int tokscan(char\* string, char\*\* next, const char\* format, ...);int tokscan(char\* string, char\*\* next, const char\* format, ...);int tokscan(char\* string, char\*\* next, const char\* format, ...);int tokscan(char\* string, char\*\* next, const char\* format, ...);int tokscan(char\* string, char\*\* next, const char\* format, ...);int tokscan(char\* string, char\*\* next, const char\* format, ...);int tokscan(char\* string, char\*\* next, const char\* format, ...);int tokscan(char\* string, char\*\* next, const char\* format, ...); $1$2Sfio\_t\* tokline(const char\* input, int flags, int\* line);Sfio\_t\* tokline(const char\* input, int flags, int\* line);Sfio\_t\* tokline(const char\* input, int flags, int\* line);Sfio\_t\* tokline(const char\* input, int flags, int\* line);Sfio\_t\* tokline(const char\* input, int flags, int\* line);Sfio\_t\* tokline(const char\* input, int flags, int\* line);Sfio\_t\* tokline(const char\* input, int flags, int\* line);Sfio\_t\* tokline(const char\* input, int flags, int\* line); ## DESCRIPTION $1$2tokopentokopentokopentokopentokopentokopentokopentokopen returns a pointer to a space separated token stream on the 0 terminated string $1$2stringstringstringstringstringstringstringstring $1$2tokreadtokreadtokreadtokreadtokreadtokreadtokreadtokread returns a pointer to the next space separated token in the token stream $1$2toktoktoktoktoktoktoktok as returned by $1$2tokopentokopentokopentokopentokopentokopentokopentokopen 0 is returned when no tokens remain. $1$2tokreadtokreadtokreadtokreadtokreadtokreadtokreadtokread temporarily modifies $1$2stringstringstringstringstringstringstringstring by inserting 0's to terminate each token. $1$2tokclosetokclosetokclosetokclosetokclosetokclosetokclosetokclose closes the token stream and restores $1$2stringstringstringstringstringstringstringstring to its original state. $1$2tokscantokscantokscantokscantokscantokscantokscantokscan scans the string $1$2stringstringstringstringstringstringstringstring for tokens specified in $1$2formatformatformatformatformatformatformatformat It is a more forgiving [*sscanf*(3)](/sscanf/3.md). If $1$2next != 0next != 0next != 0next != 0next != 0next != 0next != 0next != 0 then it will point to the next unread character in $1$2stringstringstringstringstringstringstringstring on return. The number of scanned tokens is returned. $1$2-1-1-1-1-1-1-1-1 is returned if $1$2stringstringstringstringstringstringstringstring was not empty and $1$2formatformatformatformatformatformatformatformat failed to match and tokens. *space* in $1$2formatformatformatformatformatformatformatformat matches 0 or more *space* or *tab* characters. *newline* in format eats the remainder of the current line in $1$2stringstringstringstringstringstringstringstring "...", '...' and \\*character* quotes are interpreted. A quoted *carriage-return* is converted to *newline*. *newline* in $1$2stringstringstringstringstringstringstringstring is equivalent to end of string except when quoted. *\\newline* is a line splice. $1$2%%%%%%%% in $1$2formatformatformatformatformatformatformatformat prefixes format conversion characters; each conversion character corresponds to a $1$2tokscantokscantokscantokscantokscantokscantokscantokscan argument following the $1$2formatformatformatformatformatformatformatformat argument. The format conversions are: - $1$2%c%c%c%c%c%c%c%c A single $1$2charcharcharcharcharcharcharchar - $1$2%hd %d %ld%hd %d %ld%hd %d %ld%hd %d %ld%hd %d %ld%hd %d %ld%hd %d %ld%hd %d %ld \[short, int, long\] base 10 integer. - $1$2%hn %n %ln%hn %n %ln%hn %n %ln%hn %n %ln%hn %n %ln%hn %n %ln%hn %n %ln%hn %n %ln \[short, int, long\] C-style base integer. - $1$2%ho %o %lo%ho %o %lo%ho %o %lo%ho %o %lo%ho %o %lo%ho %o %lo%ho %o %lo%ho %o %lo \[short, int, long\] base 8 integer. - $1$2%s%s%s%s%s%s%s%s String. - $1$2%hu %u %lu%hu %u %lu%hu %u %lu%hu %u %lu%hu %u %lu%hu %u %lu%hu %u %lu%hu %u %lu \[short, int, long\] C-style base unsigned integer. - $1$2%v%v%v%v%v%v%v%v The next two arguments are a pointer to a $1$2char\*\*char\*\*char\*\*char\*\*char\*\*char\*\*char\*\*char\*\* argument vector and the maximum number of elements in the vector. - $1$2%hx %x %lx%hx %x %lx%hx %x %lx%hx %x %lx%hx %x %lx%hx %x %lx%hx %x %lx%hx %x %lx \[short, int, long\] base 16 integer. $1$2%s%s%s%s%s%s%s%s and $1$2%v%v%v%v%v%v%v%v data may also be counted length strings of the form (*count*:*data*) where *count* is the number of characters in *data* and the terminating $1$2)))))))) may also be a *tab*, or the data may be $1$2(null)(null)(null)(null)(null)(null)(null)(null) which represents the $1$2NULLNULLNULLNULLNULLNULLNULLNULL string. $1$2toklinetoklinetoklinetoklinetoklinetoklinetoklinetokline returns an [*sfio*(3)](/sfio/3.md) stream to a file or string that splices *\\newline* into single lines, allows "..." and '...' to quotes to span *newlines* (done by translating quoted *newline* to *carriage-return*; $1$2tokscantokscantokscantokscantokscantokscantokscantokscan above converts quoted *carriage-return* back to *newline*), and deletes *# ... newline* comments. This is done by pushing an *sfio* discipline onto a string or file stream. Seeks are disabled on the resulting stream. If $1$2flags == SF\_READflags == SF\_READflags == SF\_READflags == SF\_READflags == SF\_READflags == SF\_READflags == SF\_READflags == SF\_READ then $1$2inputinputinputinputinputinputinputinput is a file name; If $1$2flags == SF\_STRINGflags == SF\_STRINGflags == SF\_STRINGflags == SF\_STRINGflags == SF\_STRINGflags == SF\_STRINGflags == SF\_STRINGflags == SF\_STRING then $1$2inputinputinputinputinputinputinputinput is a 0 terminated string; otherwise $1$2inputinputinputinputinputinputinputinput is an open $1$2Sfio\_t\*Sfio\_t\*Sfio\_t\*Sfio\_t\*Sfio\_t\*Sfio\_t\*Sfio\_t\*Sfio\_t\* stream. If $1$2line != 0line != 0line != 0line != 0line != 0line != 0line != 0line != 0 then it points to a line count that is initialized to 0 and is incremented for each input line. ## SEE ALSO [sfio(3)](/sfio/3.md)