Comment by norir
4 days ago
When I run a profiler on a compiler I wrote (which parses at somewhere between 500K-1MM lines per second without a separate lexer), parsing barely shows up. I'd be very surprised if the zig compiler is spending more than 5% of the time tokenizing.
I assume there is some other use case that is motivating this work.
I imagine it would be quite useful for building a responsive language server, where parsing is a more significant portion of the work
No, the problem for a language server is incremental performance, not batch performance. Although there are a lot of bad implementations out there that just reparse the entire buffer on each edit (without the error recovery benefits an incremental parser would give you).
> No, the problem for a language server is incremental performance, not batch performance
"When something is fast enough, people start to use it differently" - Linus Torvalds.
Make your parser able to parse the current file at 30FPS and you do not need incremental parsing anymore nor error recovery. That is probably part of the idea here.
4 replies →