The "knowledge" of an LLM is indeed stored in the connections between neurons. This is analogous to real neurons as well. Your neurons and the connections between them is the memory.
I'm not aware that (base) LLMs use any form of database to generate their answers- so yes, all their knowledge is stored in their hundreds of billions of synapses.
Hm. I've always commented on my (temporarily) non-retrievable memories as, "The data is still in there, it's the retrieval mechanism that degrades if not used." And, sure enough, in most cases the memory returns in a day or so, even if you don't think hard about it. (There are cases where the memory doesn't come back, as if it was actively erased or was never in long term memory in the first place. Also, as I pass eighty, I find it increasingly difficult to memorize things, and I forget recent events more readily. But I remember decades old events about as well as I ever did.)
So, my first response to your comment about the memory not being in the synapses was to agree with you. But I also agree with your respondent, so, hm.
The "knowledge" of an LLM is indeed stored in the connections between neurons. This is analogous to real neurons as well. Your neurons and the connections between them is the memory.
I'm not aware that (base) LLMs use any form of database to generate their answers- so yes, all their knowledge is stored in their hundreds of billions of synapses.
Fair enough. OTOH, generating human-like text responses is a relatively small part of the human brain's skillset.
Hm. I've always commented on my (temporarily) non-retrievable memories as, "The data is still in there, it's the retrieval mechanism that degrades if not used." And, sure enough, in most cases the memory returns in a day or so, even if you don't think hard about it. (There are cases where the memory doesn't come back, as if it was actively erased or was never in long term memory in the first place. Also, as I pass eighty, I find it increasingly difficult to memorize things, and I forget recent events more readily. But I remember decades old events about as well as I ever did.)
So, my first response to your comment about the memory not being in the synapses was to agree with you. But I also agree with your respondent, so, hm.
I don't know - it's about the best I can manage some days...