Comment by imiric
7 days ago
> They’re very good at honing bad code into good code with good feedback.
And they're very bad at keeping other code good across iterations. So you might find that while they might've fixed the specific thing you asked for—in the best case scenario, assuming no hallucinations and such—they inadvertently broke something else. So this quickly becomes a game of whack-a-mole, at which point it's safer, quicker, and easier to fix it yourself. IME the chance of this happening is directly proportional to the length of the context.
This typically happens when you run the chat too long. When it gives you a new codebase, fire up a new chat so the old stuff doesn't poison the context window.
But it rarely gives me a totally-new codebase unless I'm working on a very small project -- so I have to choose between ditching its understanding of some parts (e.g. "don't introduce this bug here, please") and avoiding confusion with others.
Why isn’t it smart enough to recognize new contexts that aren’t related to old ones?
I don't know, I didn't invent transformers. I do however know how to work with them.