Monday, October 10, 2005

Lexer state

Lexer state cannot be avoided when implementing a lexer for Perl and Ruby.

In Perl/Ruby, regular expression starts and ends with '/', for example:


As we all know, '/' is the divide operator.

So when the lexer meets a '/' character, it can not tell whether it is the start of a regular expression or a divide operator without the help of the context.

Same thing for "<<", it can be the left shift operator, or the start of "here doc":



Post a Comment

<< Home