lexer_rust.ts

Hand-written Rust lexer.

A single flat scan loop — Rust has no re-entrant interiors at the token level (no template interpolations), so the constructs that nest are handled with counters in one forward pass: block comments nest by depth, raw strings close on a " followed by their opening hash count, and attribute interiors lex inline (marker and leading path attribute-styled, arguments under the normal rules) with a [/] depth counter marking where the bracket closes.

Disambiguation notes: ' is a lifetime when ident-shaped text follows without a closing quote ('a, 'static, loop labels) and a char literal otherwise; name! is a macro invocation unless the ! starts !=; the string prefixes (r/b/c/br/cr) are recognized before generic identifier scanning and fall through to it when no quote follows.

Resilience: unterminated block comments, strings (which span lines in Rust), and raw strings extend to the end of the window; an unterminated attribute simply lexes its interior to there.