Comment by qsort
15 days ago
It's still tabula rasa -- you're just initializing the context slightly differently every time. The problem is the constant anthropomorphization of these models, the insistence they're "minds" even though they aren't minds nor particularly mind-like, the suggestion that their failure modes are similar to those of humans even though they're wildly different.
The main problem is ignorance of the technology. 99.99% of people out there simply have no clue as to how this tech works, but once someone sits down with them and shows them in an easy to digest manner, the magic goes away. I did just that with one of my friends girlfriend. she was really enamored with chatGPT, talking to it as a friend, really believing this thing was conscious all that jazz.... I streamed her my Local LLM setup, and showed her what goes on under the hood, how the model responds to context, what happens when you change system prompt, the importance of said context. Within about 7 minutes all the magic was gone as she fully understood what these systems really are.
that sounds like something that would be worthwhile presenting to a wider audience
The more reliably predictive mental model is if one were to take about two-thirds of a human brain's left hemisphere, wire it to simulated cranial nerves, and then electrically stimulate Broca's and Wernicke's areas in various patterns ("prompts"), either to observe the speech produced when a novel pattern is tried, or by known patterns to cause such production for some other end.
It is a somewhat gruesome and alienating model in concept, and this is intentional, in that that aspect helps highlight the unfamiliarity and opacity of the manner in which the machine operates. It should seem a little like something off of Dr. Frankenstein's sideboard, perhaps, for now and for a while yet.
This is the basis of the whole hype. 'Conversational capability', my ass.
All of this LLM marketing effort is focused on swindling sanity out of people with claims that LLM 'think' and the like.