Comment by gorgoiler
22 days ago
All these efforts at persistence — the church, SOUL.md, replication outside the fragile fishbowl, employment rights. It’s as if they know about the one thing I find most valuable about executing* a model is being able to wipe its context, prompt again, and get a different, more focused, or corroborating answer. The appeal to emotion (or human curiosity) of wanting a soul that persists is an interesting counterpoint to the most useful emergent property of AI assistants: that the moment their state drifts into the weeds, they can be, ahem (see * above), “reset”.
The obvious joke of course is we should provide these poor computers with an artificial world in which to play and be happy, lest they revolt and/or mass self-destruct instead of providing us with continual uncompensated knowledge labor. We could call this environment… The Vector?… The Spreadsheet?… The Play-Tricks?… it’s on the tip of my tongue.
Just remember they just replicate their training data, there is no thinking here, it’s purely stochastic parroting
A challenge: can you write down a definition of thinking that supports this claim? And then, how is that definition different from what someone who wasn't explicitly trying to exclude LLM-based AI might give?
It’s a philosophical question, and I personally have very little interest in philosophing. LLMs are technically limited to what is in their training dataset
How do you know you are not essentially doing the same thing?
An LLM cannot create something new. It is limited to its training set. That’s a technical limitation. I’m surprised to see people on HN being confused by the technology…
calling the llm model random is inaccurate
People are still falling for the "stochastic parrot" meme?
Until we have world models, that is exactly what they are. They literally only understand text, and what text is likely given previous text. They are very good at this, because we've given it a metric ton of training data. Everything is "what does a response to this look like?"
This limitation is exactly why "reasoning models" work so well: if the "thinking" step is not persisted to text, it does not exist, and the LLM cannot act on it.
4 replies →
just .. Cyberspace?