Comment by sly010
6 years ago
> The only reason an external file/dag is actually necessary is due to impurity in the compilation process.
But files also make various pieces of compiler chain interoperable and allows me to define a DAG. That's exactly what make is so powerful and that's exactly what I'd hate to loose.
Modern compilers do a lot and understandably they are trying to avoid writing out some partially calculated state to disk (e.g. serializing and AST to disk between stages would be doing work twice). But moving everything into the process means your compiler becomes a walled garden.
You can see this happening in the javascript world. Very few people actually know what WebPack does. It's a giant black box with infinite number of switches and everything is "magic".
I totally agree with this and think it's a valid concern, It difficult to get the best of both worlds here within the confines of the unix process model.
The query style compiler isn't by necessity a single process. You could imagine an implementation based on the actor model or almost any other message passing system where the queries are driven externally. That the expedient way to do this is by stuffing everything into a giant process is regrettable.