`xtokid' accepts the names of files and/or directories on the command line, then extracts and prints a stream of tokens from those files for which it has a valid, enabled scanner. This is useful primarily for debugging new `mkid' scanners (see section Defining New Scanners in the Source Code).
`xtokid' extracts tokens from source files, therefore it accepts the `--lang-map', `--include', `--exclude', and `--lang-option' options, as well as the language-specific scanner options, all of which are described in section Options for Programs that Scan Source Files. `xtokid' walks file trees, therefore it handles file and directory names on its command line and the `--prune' option as described in section Options for Programs that Walk File and Directory Trees..
The name `xtokid' indicates that it is the "eXtract TOKens ID utility".
Go to the first, previous, next, last section, table of contents.