Comment by mshockwave
6 years ago
Good...but why does the author think modern compilers / language servers DON'T do caching? Or in another way: why does the author think caching mechanism in modern compiler / language server is insufficient? I think the author is proposing a caching mechanism that has finer granularity and design the whole system around this idea from day one. But first, at least many components in LLVM have been doing caching for a long time (e.g. OrcJIT has a pretty decent caching layer and libTooling also supports incremental parsing with AST caches). Second, what is the memory overhead of this (fine grain caching) design when it's facing some larger input program in real world (e.g. OS kernel)? Does it scale well?
I know it's refurbished from an old post so I probably shouldn't be so harsh. But it will be better to compare some old ideas against state-of art works and found some insights rather than doing pure archaeology
I am not the author, but work in the same domain. Empirically, existing compilers are impossible to turn into good IDEs, unless the language has header files and forward declarations. Otherwise, you get a decent IDE by doing one of the following:
* writing two compilers (C#, Dart, Java(and, in some sense, every major language, supported in JetBrains tooling)
* starting with IDE compiler from the start (Kotlin & TypeScript)
The two examples where I’ve heard a batch compiler was successfully used to build a language server are C++ and OCaml (haven’t tried these language servers myself though). Curiously, they both use header files, which means that it’s the user who does fine grained caching.
I don‘t see how caching in LLVM is relevant to the task of building LSP servers.
In terms of prior art, I would suggest studying how IntelliJ works, specifically looking at the stubs:
https://www.jetbrains.org/intellij/sdk/docs/basics/indexing_...
There are ways to turn batch compilers into incremental IDE compilers with some tree and effect caching on the periphery of the compiler. You can even go all the way up to the eval level for a full live programming environment. You don’t need module signatures if you are willing to trace dependencies dynamically.
See https://www.microsoft.com/en-us/research/publication/program....
> OrcJIT has a pretty decent caching layer
(Sadly) this is not correct. If you want to cache compiled code between sessions, your only option so far are handwritten object caches along the lines of this example:
https://github.com/llvm/llvm-project/blob/ae47d158a096abad43...