Comment by gustavopezzi

Lua's LPeg library would also be a great option here. A couple of years ago I taught an interpreters class together with Roberto Ierusalimschy and we used LPeg for lexing and parsing. The end result was great. Even if you're not using PEGs for your implementation I would recommend spending 30 minutes and looking at LPeg.