Comment by pona-a
6 days ago
> It’s projection. People say “LLMs can’t code” when what they really mean is “LLMs can’t write Rust”. Fair enough! But people select languages in part based on how well LLMs work with them, so Rust people should get on that.
How is it the responsibility of the Rust community that there weren't enough metric tons of free code for the machine to slurp up? And the phrasing makes it sound like it's the community's fault for not feeding OpenAI enough code to be stripped of its license and authorship and get blended into a fine latent soup. It's a lot like people coming to a one-man FOSS project with a laundry list of demands, expecting to be treated with the religious reverence of a major enterprise contract.
The whole tone, the pervasive "use it or you'll be left behind"—where users saying they don't want or need it only proves further evidence of its imminent apotheosis—superficially reminds me of previous FUDs.
And how is it not concerning that the thing described as intelligent needs billions of lines to generalize a language a human can learn from a single manual? Will it need hundreds of kLOC to internalize a new library, or even its new version, beyond in-context learning? The answer is yes; you are choosing to freeze the entire tech stack, when fixing its abstractions could actually save you from boilerplate, just so the machine can write it for you at $200 a month with a significant error rate.
Is the problem with LLMs and Rust really a problem with the training data or a problem with it being harder to get something to compile in Rust? I presume its easier to get something to compile in the dynamically typed languages, and then slightly more difficult in statically typed languages and then a step up with Rust which also forces you to deal with the lifetime of objects. But maybe with LLMs its different and they can do static typing easier than dynamic but they just really struggle with structuring a program to properly deal with ownership.
They mostly copy and mix patterns in the training data. Lots of repetition with variations on them is helpful for their generalization. Languages like Python and Go have a ton of code in Github, etc like that. I saw that using Python with GPT 3.5/4.
If it's a rarer language, the math doesn't do as good of a job on piles of random code. There's just not enough for it to learn from. I cant speak for Rust since I dont know the numbers but imagine it's much less than Python or Go.
I have seen some evidence, though, that harder languages are harder for them to code in. GPT 3.5 used to struggle with C++ for something that it could easily produce in Python. It could actually produce things in C more easily than C++. It makes sense, though, because there's both more context needed for correctness and more behavioral patterns to write it.
My solution, which I only prototyped in GPT due to leaving AI, was to use AI's to write code in languages like Python which non-AI tools transpiled to high-performance code in C++ or Rust. Think the Python to C++ compiler or maybe Nikita. Later, with hallucinations mitigated enough, add LLM's to those transpilers.
As a side benefit, it let's you sell a product accelerating or increasing predictability of applications in that language. That's a non-AI investment. There's a few companies doing that, too. So, one could sell to the AI crowd, the "language X in business" crowd, or both.