Comment by LoveMistral

Yes - you are experiencing a mix of context caching and db retrieval from these mainstream model experiences.

Even the best models available lose a ton of detail over time if you were to paste in tens of thousands of lines of code.

The only way to hold huge amounts of context with a high degree of accuracy is to store it using various mechanisms (one of which is RAG).

On “effectiveness”, I mean end use case effectiveness in the tasks at hand, not whatever benchmark the model developer or vendor themselves come up with - which may or may not be useful to the work I’m doing.