TokenStructure

The token that is returned by the lexer.

struct TokenStructure (

IdType

string extraFields = ""

) {

bool opEquals(IdType type);

this(IdType type);

this(IdType type, string text, size_t line, size_t column, size_t index);

pure nothrow @safe @nogc

pure nothrow @safe @nogc

pure nothrow @safe @nogc

size_t column;

pure nothrow @safe @nogc

size_t index;

pure nothrow @safe @nogc

}

Constructors

this this(IdType type): Constructs a token from a token type.
this this(IdType type, string text, size_t line, size_t column, size_t index): Constructs a token.

Members

Functions

opEquals bool opEquals(IdType type): Returs: true if the token has the given type, false otherwise.

Variables

column size_t column;: The column number at which this token occurs. This is measured in bytes and may not be correct when tab characters are involved.
index size_t index;: The byte offset from the beginning of the input at which this token occurs.
line size_t line;: The line number at which this token occurs.
text string text;: The text of the token.
type IdType type;: The token type.

Examples

// No extra struct fields are desired in this example, so leave it blank.
alias Token = TokenStructure!(IdType, "");
Token minusToken = Token(tok!"-");

Meta

Source

See Implementation

std experimental lexer functions

tokenStringRepresentation

mixin templates

Lexer

structs

templates