TOK

< Token kind. TODO: make this a string type like with std.experimental.lexer

Values

ValueMeaning
none
unknown

< Unknown

whitespace

< Whitespace

symbol

< Symbol

attributeSymbol

< Attribute Symbol (starting with $)

actionSymbol

< Action Symbol (starting with @)

number

< Number

lineComment

< Single line comment

blockComment

< Multi-line (block) comment

leftParen

< Left parenthesis

rightParen

< Right parenthesis

action

< Code block

brackets

< Alternatives within '[' ... ']'

literal

< Text literal, single or double quoted

colon

< Colon :

semicolon

< Semicolon ;

hash

< Hash #

labelAssignment

< Label assignment =

listLabelAssignment

< List label assignment +=

qmark

< Greedy optional or semantic predicate (?)

qmarkQmark

< Non-Greedy optional (??)

star

< Greedy zero or more (*)

starQmark

< Non-Greedy Zero or more (*?)

plus

< Greedy one or more (+)

plusQmark

< Non-Greedy One or more (+?)

pipe

< Alternative (|)

tilde

< Match negation (~)

lt

< <

gt

< >

comma

< .

exclamation

< Exclude from AST (!)

rootNode

< Root node (^)

wildcard

< .

dotdot

< ..

rewrite

< Rewrite rule (->)

rewriteSyntacticPredicate

Syntactic predicate rule rewrite (=>).

Wikipedia: A syntactic predicate specifies the syntactic validity of applying a production in a formal grammar and is analogous to a semantic predicate that specifies the semantic validity of applying a production. It is a simple and effective means of dramatically improving the recognition strength of an LL parser by providing arbitrary lookahead. In their original implementation, syntactic predicates had the form “( α )?” and could only appear on the left edge of a production. The required syntactic condition α could be any valid context-free grammar fragment.

See_Also: https://en.wikipedia.org/wiki/Syntactic_predicate See_Also: https://wincent.com/wiki/ANTLR_predicates

tokenSpecOptions

Token spec options: "<" id ASSIGN optionValue ( SEMI id ASSIGN optionValue )* ">" ;

_error

< Error token

Meta