Comment by stared

17 hours ago

There was a recent discussion, “Why AI Needs Hard Rules, Not Vibe Checks” (https://news.ycombinator.com/item?id=46152838). We need as many checks as possible - and ideally ones that come for free (e.g., guaranteed by types, lifetimes, etc.) - which is why Rust might be the language for vibe coding.

Without checks and feedback, LLMs can easily generate unsafe code. So even if they can generate C or Assembly that works, they’re likely to produce code that’s riddled with incorrect edge cases, memory leaks, and so on.

Also, abstraction isn’t only for humans; it’s also for LLMs. Sure, they might benefit from different kinds of abstraction - but that doesn’t mean “oh, just write machine code” is the way to go.

That's a really, really interesting point.

It makes me imagine a programming language designed for LLMs but not humans, designed for rigorous specification of every function, variable, type, etc., valid inputs and outputs, tightly coupled to unit tests, mandatory explicit handling of every exception, etc.

Maybe it'll look like a lot of boilerplate but make it easy to read as opposed to easy to write.

The idea of a language that is extremely high-effort to write, but massively assists in guaranteeing correctness, could be ideal for LLM's.

  • That’s what the article is about.

    • No it's not. The article proposes the idea of a language designed for vibe-coding, and suggests several variants designed for specific purposes. But none of the variants are for the purpose I suggested, which is about maximizing correctness. That's the point I was making.

  • I'm writing one of these, I'll post it on HN next year. The key to a language for LLMs is: make sure all the context is local, and explicit. If you have functions, use parameters for arguments instead of positions. If you have types, spell them out right there. Also, don't use too many tokens, so keywords are out. And that's just a start.

    I think the ideal language for LLMs will look more like APL than C.

If it needs that many rules, why use AI at all? Linters pre-exist AI, don't cost money, and don't boil the oceans.

Look at Shellcheck. It turns a total newbie into a shell master just by iteration.

Absolutely. A language being well suited to static analysis and "compiler driven development" matters a lot more with LLMs than with humans IMO

We're at the point of diminishing returns from scaling and RL is the only way to see meaningful improvements

Very hard to improve much via RL without some way to tell if the code works without requiring compilation

Logic based languages like Prolog take this to the logic extreme, would love to see people revisit that idea

Rust doesn't prevent programs from having logic errors.

If LLMs produce code riddled with bugs in one language it will do in other languages as well. Rust isn't going to save you.

  • Idiomatic Rust prevents many classes of logic errors. Just having proper sum types eliminates many (perhaps most) common logic errors.

  • > Rust doesn't prevent programs from having logic errors.

    Like everything around Rust, this has been discussed ad nauseam.

    Preventing memory safety bugs has a meaningful impact in reducing CVEs, even if it has no impact on logic bugs. (Which: I think you could argue the flexible and expressive type system helps with. But for the sake of this argument, let's say it provides no benefits.)

    • It isn't like rust is the only language with memory safety; plenty of high level languages don't let you fiddle with memory bits in a way that would be unsafe. The tradeoff is that they typically come with garbage collectors.

      If the only concern is "can an LLM write code in this language without memory errors" then there's plenty of reasons to choose a language other than Rust.

      5 replies →

  • All kinds of drugs produce unwanted risks and side effects if abused, so let's abuse crystal meth! Cannabis isn't going to save you.

  • > Rust doesn't prevent programs from having logic errors.

    Nobody ever claimed that. The claims are:

    1. Rust drastically reduces the chance of memory errors. (Or eliminates them if you avoid unsafe code.)

    2. Rust reduces the chance of other logic errors.

    Rust doesn't have to eliminate logic errors to be a better choice than C or assembly. Significantly reducing their likelihood is enough.

    • Can these claims back themselves up with a study showing that over a large population size with sufficient variety, sourced from a collection of diverse environments, LLM output across a period of time is more reliably correct and without issue when outputting Rust? Otherwise this is nothing but unempirical conjecture.

      3 replies →

  • This is objectively wrong.

    You can't get a gutter ball if you put up the rails in a bowling lane. Rust's memory safety is the rails here.

    You might get different "bad code" from AI, but if it can self-validate that some code it spits out has memory management issues at compile time, it helps the development. Same as with a human.

    • > You can't get a gutter ball if you put up the rails in a bowling lane.

      Sure you can. It's difficult, and takes skill, but it can be done.

  • Modern medicine can't prevent or cure all diseases, so you might as well go back to drinking mercury then rubbing dog shit into your wounds.

    Modern sewers sometimes back up, so might as well just releive yourself in a bucket and dump it into your sidewalk.

    Modern food preservation doesn't prevent all spoilage so you might as well just go back to hoping that meat hasn't been sitting in the sun for too many days.

  • > Rust doesn't prevent programs from having logic errors.

    Sure, but it prevents memory safety issues, which C doesn't. As for logic bugs, what does prevent them? That's a bigger question but I'd suggest it's:

    1. The ability to model your problem in a way that can be "checked". This is usually done via type systems, and Rust has an arguably good type system for this.

    2. Tests that allow you to model your problem in terms of assertions. Rust has decent testing tooling but it's not amazing, and I think this is actually a strike against Rust to a degree. That said, proptest, fuzzing, debug assertions, etc, are all present and available for Rust developers.

    There are other options like using external modeling tools like TLA+ but those are decoupled from your language, all you can ever do is prove that your algorithm as specified is correct, not the code you wrote - type systems are a better tool to some degree in that way.

    I think that if you were to ask an LLM to write very correct code then give two languages, one with a powerful, express type system and testing utilities, and one without those, then the LLM would be far more likely to produce buggy code in the system without those features.

    • Logic errors always stems from lack of understanding and inattention. The former is resolved by good communication and analytical skills. The other is just human nature, but we do have guardrails to help, like static analysis and tests. If used correctly.

      There are static tools available for C as well. What you get from Rust mostly is that the check is part of the syntax of the language as well and escaping from it is very visible. You get safety, but you give up flexibility and speed.

Why Rust? Haskell is gold standard here.

  • Can you elaborate? What is it about Haskell that makes it better?

    • Very advanced type system which allows to move a lot of program correctness to typing system. So basically if your program compiles, it probably works.

      It's also has GC which makes it better suited for most programs, compared to Rust with its manual memory management.

      1 reply →

    • Purely functional code is easier to test because of its referential transparency and lack of shared state.

      Haskell is also nice because of quickcheck.

  • I would think Lean and other formal languages are the real gold standard.

    But none of them really have enough training data for LLMs to be any good at them.

I think it's a pretty good point. I've been using LLMs for .NET and the output is generally pretty good.