Comment by BoorishBears

It really funny reading the reporting on this because everyone is (very reasonably) thinking Replit has an actual 'code freeze' feature that the AI violated.

Meanwhile by 'code freeze' they actually meant they had told the agent that they were declaring a code freeze in natural language and I guess expected that to work even though there's probably a system prompt specifically telling it its job is to make edits.

It feels a bit like Michael from The Office yelling "bankruptcy!"

-

I have to say, instruction tuning is probably going to go down in history as one of the most brilliant UX implementations ever, but also has had some pretty clear downsides.

It made LLMs infinitely more approachable than using them via completions, and is entirely responsible for 99% of the meteoric rise in relevance that's happened in the last 3 years.

At the same time, it's made it painfully easy to draw completely incorrect insights about how models work, how they'll scale to new problems etc.

I think it's still a net gain because most people would not have adapted to using models without instruction tuning... but a lot of stuff like "I told it not to do X and it did X" where X is something no one would expect an LLM to understand by its very nature, would not happen if people were forced to have a deeper understanding of the model before they could leverage it.