Comment by rhdunn
5 hours ago
I've not written a language server but have written a language plugin for IntelliJ.
I started with writing a correct recursive descent parser. I then extended it to detect, report, and recover from common syntax errors as I encountered them so that the parser is robust. And adding a parser test case for each of these (e.g. one test for each branch through an EBNF construction).
Some examples are:
1. missing keywords when the keyword can be detected from the current context (e.g. missing semicolon at the end of a statement);
2. using the wrong token (e.g. `:` instead of `::` in a C++ namespace qualified name);
3. detecting and ignoring whitespace in a whitespace-sensitive qualification (e.g. in XML QNames);
4. keeping in the prolog state (where functions are defined) when there are errors so that functions after the error don't get lost;
5. lexing incomplete literals like `10e` so they can be handled as integers in the parser and emitting an error for them.
> recover from common syntax errors
It's a dead-end. Sure, it can work in simple cases, but there will be always a case where such syntax recovery isn't possible. That's why relying only on syntax recovery isn't an option.
Because of that I use a different approach. I do parse on each document editing, but such parsing is guaranteed to produce valid results only up to the point with broken syntax, where editing usually takes place. Such parsing is enough to reconstruct location of the point where editing takes place (namespace/class/function) and to reconstruct local context (local variables declared prior to editing place). This allows to perform almost perfect autocompletion by suggesting global and local names available at the editing point. In order to provide proper suggestion of non-local names declared after the editing point, I do keep a structure for the most recent document state with valid syntax.
With features like "go to definition" I do the same. I store a hash-table with location to definition point mapping, but it's updated only from time to time and only if document syntax is valid. In order to be usable for cases with edits made after building such hash-table I just perform text-based position mapping using accumulated edit events.