Comment by jrflowers
15 days ago
>IF you did this SAME experiment with a human and had a human read an ENTIRE document and then reproduce said document with edits. The DOCUMENT would DEGRADE even more.
I like the idea that imagining somebody doing something in a way that nobody does it because it makes no sense for a person to do it like that is helpful here. It is like
IF you made a human eat an ENTIRE IHOP™ Chicken Fajita Omelette in one bite they would CHOKE and the OMELETTE would go UNDIGESTED. It would get everywhere and the OMELETTE would be RUINED.
That's the point bro. I am saying this Experiment makes no sense.
Humans don't do that. And Claude doesn't edit documents like that. Because it makes no sense. The point is saying that the Experiment itself is not helpful here.
It is, in fact, pretty common for people to throw a document at a language model along with a “make it more gooder” prompt.
That was true maybe 7 months ago. This is no longer the case. Harnesses use all kinds of tooling to edit things now.
11 replies →