Comment by _bobm

This is true, but consider that we have agentic token injection (what you are referring, prompting), embedding injection (memorilla) and continual learning.

So the first will update _what_ information the model reasons/infers about, the last will update _how_ the model reasons/ infers about.

These are very different.

As far as I understand memorilla sits in the middle. And it embeds not tokens but embedding vectors with all their complexity. The compression (semantic) is significant and in my mind is super interesting.