← Back to context

Comment by janalsncm

21 hours ago

A bit too pessimistic imo. I agree that AI can’t automate things end to end, but a good deal of R&D involves kicking off a training run and babysitting it.

If your training run dies at 1 am and you’re sleeping, you won’t find out about it until the next day. You can lose up to 18 hours of work depending on when it happens. Based on the error it might be as simple as tweaking a single hyperparameter and rebooting, which is something LLMs are usually capable of.

Even just that task means I can kick off multiple runs over the weekend and have confidence they’ll finish. It’s a game changer.

4 comments

janalsncm

Reply
rmunn  21 hours ago

I'd classify that as an entirely different category than AI self-training. What you're describing could have been done with a short script, though which parameter to tweak and how to tweak it would be difficult to automate with a non-LLM script, so the LLM's being able to parse the error message and base the tweak on the content of the error is a definite improvement to the process there.

But I'd classify this as LLM being used to automate a sysadmin task, rather than calling that self-training.

  • janalsncm  20 hours ago

    Yeah I’m not trying to argue it is AGI, but it’s not as simple as a short script. There’s some amount of debugging involved, and no amount of if-statements could cover all possible ways a script could break.

    In a way, “recursive self improvement” just means tools helping us to create better tools. At least that’s what the words mean.

    • cyanydeez  11 hours ago

      the GP posits the problem with "recursive self improvement" is the poisoned context & hallucination problem. While a strong loop that has fail safes, backups, restore points (essentially, a fancy backup system), the problem isn't that we cannot create a healthy advanced wiggum loop; it's that every step of the LLM as it grows whatever knowledge is acretes, has a chance of being either poisoned (eg, it conflates two tokens as describe different things) or wholesales fabricates a method or procedure.

      Now humans are just as bad, but they're not moving at the speed of compute so the posion and fabricates can dissolve over time, or just, as you've noticed turning on your news, get stuck in very stupid positions. So humans are clearly capable but clearly don't tend to do this either.

      So then we dont have a real road map. The error rates, although small, acrete at exponential levels and will wash out improvements.

      So I also had the idea that "if we just give it enough context, surely it'll be more powerful". But the error rates hit that squarely. The larger the context grows, the more likely it hasn't properly organized its knowledge to avoid overlapping facts.

      In programming, it's worse, because a lot of the code is purposefully "DRY" and reuseable. Everye C program has a main(); is it remembering the correct main? or any of the number of same variables?

      You can see an LLM is powerful but it's not ominipotent. It'll suffer very much when it starts hallucinations and context poisoning.

      So, sure you can try a super ralph wiggum loop with memory, fallback safeties, etc, but you basically then need another turtle that does the same thing, and at that point, you're positing a infinite jest of ralph wiggum loops tracking each other, recursively, forever.