Comment by sornaensis
16 hours ago
I'm curious what issues you had with haskell? I have had the opposite experience and find them dreadful at Java et al.
Surely, denser languages should be better for LLMs?
16 hours ago
I'm curious what issues you had with haskell? I have had the opposite experience and find them dreadful at Java et al.
Surely, denser languages should be better for LLMs?
The context window also limits how deeply the model can "think", and it does this in natural language. So a language suited to LLMs would have balanced density, if it's too dense, the model spends many tokens working through the logic, if it's too sparse, it spends many tokens to read/write the code.
I think in the context of already trained LLMs, the languages most suited to LLMs are also the ones most suited to humans. Besides just having the most code to train on, humans also face similar limitations, if the language is too dense they have to be very careful in considering how to do something, if it's too sparse, the code becomes a pain to maintain.
I generally agree that humans and LLMs benefit similarly from programming language features. I would tweak that a bit and suggest that their ability floor is higher than the human lowest common denominator so I would skew towards the more advanced human programming languages. There are many typing / analyzer features that would be frustrating for humans to use given they’ll cause the type checking to be slower. This is much less of a problem for LLMs in that they’re very patient and are much better at internalizing the type system so they don’t need to trigger it anywhere nearly as often.
Density is a double edged sword. On the one hand you want to minimise context usage, but on the other hand more text on the page means more that the LLM can work with.
my (uninformed) speculation is that you want resilience and error correction, which implies some level of redundancy rather than pure density.