Comment by Anamon

3 days ago

This would be fine if LLMs generated quality code, which they don't. Anything beyond trivial and boilerplate code is either riddled with errors or copied almost verbatim. None of these systems are able to even remotely do what a competent developer does.

Despite the PR author's claims, LLMs have no, and can't have any, understanding of the code. Especially when you start talking about architecture, robustness, security, etc. And those are the really challenging parts. Coding is 10% of a developer's job, and they're usually the easiest. If reasonably used LLM tools can help developers code, awesome. But that part was never the problem or the bottleneck.

The chess/Go analogy doesn't work, because those are games that have set rules and winning conditions. Algorithms can work with that, that's why they beat humans. The "winning conditions" of software development are notoriously tricky to get right an often impossible to perfectly formulate. If they weren't, natural language programming might be a viable path. Dijkstra knew in the 70s that it can't be.[1]

Generated code can already not be ignored, but I don't think it's for the reasons implied. Someone here mentioned Brandolini's Law. We can't ignore it for the same reason we can't ignore spam e-mails. They're too easy and cheap to produce, and practically none of what's produced has any real value or quality. We can't ignore the code because it's threatening to make an already worrying crisis of QA and security in software development even worse.

[1] https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...

3 comments

Anamon

wagerlabs 1 day ago

This is an excerpt from the session where AI is writing my Lisp compiler. What do you call this? I call this doing what a competent developer does!

39/40 tests pass. The native reader works for integers, hexadecimal, lists, strings and quote forms. The one failure is symbol comparison (known limitation).

  Based on the context summary and the user's note about cleanup, I should:
  1. Continue Phase 3.5 self-compilation
  2. Clean up the repo structure

  Let me first update the todo list and then investigate the SIGKILL issue more
  thoroughly. The issue is that combining reader source with file I/O code causes
   the executable to be killed. Let me check if buffer-to-string with reader
  works:

 Let me test specifically reader + file I/O combined (which is what the
  self-hosting test needs):

q3k 1 day ago
Sounds to me like someone roleplaying being a developer. Never in my career have I seen someone think/reason/act like this.
- cesarb 20 hours ago
  
  > Sounds to me like someone roleplaying being a developer.
  That's because that's precisely how LLMs work. They complete a text where two actors (the "user" and the "assistant"), or sometimes three actors (the "user", the "assistant", and the "tools"), are engaging in a conversation (in this case, about software development). It's like a theatre script.