syntax_token.ts

Declarations
#

2 declarations

The type of the token.

This is usually the key of a pattern in a Grammar.

type string

The strings or tokens contained by this token.

This will be a token stream if the pattern matched also defined an inside grammar.

type string | SyntaxTokenStream

The alias(es) of the token. Always an array, even if empty or single value.

type Array<string>

type number

type new (type: string, content: string | SyntaxTokenStream, alias: string | string[] | undefined, matched_str?: string): SyntaxToken

type string

type string | SyntaxTokenStream

type string | string[] | undefined

type string

default ''

SyntaxTokenStream

A token stream is an array of strings and SyntaxToken objects.

Syntax token streams have to fulfill a few properties that are assumed by most functions (mostly internal ones) that process them.

1. No adjacent strings. 2. No empty strings.

The only exception here is the token stream that only contains the empty string and nothing else.