Comment by aezart

I think you're right. When I was experimenting with llama 1, I was able to easily observe that with a short prompt and a long response, the response _rapidly_ degraded the longer it went, because it was seeing and amplifying the patterns in its context window so far.

It is intuitively obvious that these problems would get even worse if the garbage output found its way into the training set, and not just into the context window.