Interface LexToken

One lexer token of the "lex" output format.

interface LexToken {
    kind: string;
    span: { end: number; start: number };
    text: string;
}

Index

Properties

Properties

kind

kind: string

The token kind — the engine's own name for it ("BitStart(1, false)", "Text(Body, false)", "Bold", …). Not a stable vocabulary.

span

span: { end: number; start: number }

Byte offsets into the source (UTF-8).

text

text: string

The source slice.