Skip to content

Introduced new escaped identifier token

thebirdsdontsing requested to merge feature/escaped-identifier-token into main

Created by: ecoulson

What?

Introduced a new escaped identifier token to remove the coupling of character sequences and escaped identifiers.

Why?

The original tokens used to implement the lexer were insufficient to cover all the tokens found within the file. For more information, see #10 (closed)

How?

To resolve this, a new EscapedIdentifierToken was introduced, and a separate lexical operation, EscapeIdentifier, was added to allow the lexer to lex character sequences and escaped identifiers separately.

Testing?

Character sequence token tests were amended to no longer check for escaped identifiers and added two test cases that verify the new EscapedIdentifierToken is lexed correctly.

Anything Else?

resolves #10 (closed).

Merge request reports