Reading that made me think how much that might be related to Elixir being very similar in syntax to Ruby. Do LLMs really differentiate between the two?
Specific studies, as the one quoted, are a long way from original real world problems.
LLMs absolutely understand and write good Elixir. I've done complex OTP and distributed work in tandem with Sonnet/Opus and they understand it well and happily keep up. All the Elixir constructs distinct from ruby are well applied: pipes, multiple function clauses, pattern matching, etc.
I can say that anecdotally, CC/Codex are significantly more accurate and faster working with our 250K lines of Elixir than our 25K lines of JS (though not typescript).
I suspect the biggest advantage Elixir has is the relative quality of the publicly available code. Approximately no one has Elixir as their first programming language, which keeps a lot of the absolute trash-tier code that we all make when first learning to program out of the training set. If you look at languages that are often people's first (Python, JavaScript, Java), only Java has an above average score. Of those three, Java's significantly more likely to be taught in a structured learning environment, compared to kids winging it with the other two.
(And Elixir's relationship to Ruby is pretty overstated, IMO. There's definitely inspiration, but the OO-FP jump is a makes the differences pretty extreme)
Having written a lot of both languages, I'd be surprised if LLMs don't get tripped up on some of Ruby's semantics and weird stuff people do with monkey patching. I also find Ruby library documentation to be on average pretty poor.
Don't assume. Empirically, they are not. (This post Feb 2026 may change in future yadda yadda)
See: autocodebench
https://github.com/Tencent-Hunyuan/AutoCodeBenchmark/tree/ma...
Reading that made me think how much that might be related to Elixir being very similar in syntax to Ruby. Do LLMs really differentiate between the two?
Specific studies, as the one quoted, are a long way from original real world problems.
Here are some thoughts on it from José Valim: https://dashbit.co/blog/why-elixir-best-language-for-ai
LLMs absolutely understand and write good Elixir. I've done complex OTP and distributed work in tandem with Sonnet/Opus and they understand it well and happily keep up. All the Elixir constructs distinct from ruby are well applied: pipes, multiple function clauses, pattern matching, etc.
I can say that anecdotally, CC/Codex are significantly more accurate and faster working with our 250K lines of Elixir than our 25K lines of JS (though not typescript).
2 replies →
I suspect the biggest advantage Elixir has is the relative quality of the publicly available code. Approximately no one has Elixir as their first programming language, which keeps a lot of the absolute trash-tier code that we all make when first learning to program out of the training set. If you look at languages that are often people's first (Python, JavaScript, Java), only Java has an above average score. Of those three, Java's significantly more likely to be taught in a structured learning environment, compared to kids winging it with the other two.
(And Elixir's relationship to Ruby is pretty overstated, IMO. There's definitely inspiration, but the OO-FP jump is a makes the differences pretty extreme)
1 reply →
Here are my thoughts:
https://m.youtube.com/watch?v=YZa5GqrZeao
Having written a lot of both languages, I'd be surprised if LLMs don't get tripped up on some of Ruby's semantics and weird stuff people do with monkey patching. I also find Ruby library documentation to be on average pretty poor.
2 replies →