Comment by joegibbs

If you use a <8gb model you can finetune it with Unsloth in an hour or so. What if the system extracts facts and summarises its own output every day to only 10,000 lines or so, and then finetunes its base model with the accumulated data and switches to run that, as a kind of simulation of long-term memory? Within the same day it could have a kind of medium-term memory via RAG and short term memory via context.