← Back to context

Comment by bonoboTP

1 hour ago

These models are outputting language that we can decode as factual claims and we can check those, and we can say it made a mistake / error or that it answered correctly. This doesn't require any squishy assertions to how it "feels" while doing it or anything like that. It's an externally observable thing.

I agree that hallucination isn't some kind of "different" operation than "normal". It's not like when a train derails and you can point to it. It just operates as normal and sometimes that yields correct factual outputs, sometimes not. You don't have to metaphysically ascribe any kind of intent to it.

I'm not sure how people conceptualize these things who weren't doing classical machine learning before all this. To me, "hallucination" is shorthand, and we know it's not like humans on drugs or something. It was used in the literature also for any kind of generative imputing of missing information from a learned prior. For example in image inpainting a GAN "hallucinates" the missing part of the image. This terminology was already used in the 2010s and probably earlier. Or in image colorization of grayscale photos, the model "hallucinates" the color information.

Then the word escaped into the mainstream and people have weird connotations about it.

3 comments

bonoboTP

Reply
Terr_  1 hour ago

> To me, "hallucination" is shorthand [...] Then the word escaped into the mainstream

One of my bugbears is when people abuse the term "Ponzi Scheme" to refer to literally anything the think is unsustainable. (As opposed to something that, at a minimum, requires someone telling factual-lies about assets.) Kind of like if folks started calling every kind of software error a "Buffer Overflow."

  • bonoboTP  1 hour ago

    I think the colloquially Ponzi Scheme is a synonym for any pyramid scheme, where the people in the smaller upper layers rely on an ever growing sequence of lower layers entering to receive their payouts, and it works as long as there are newer people to bring in but at some point the recruitment pipeline is exhausted and the bottom layer ends up losing their entire investment and the higher you are in the pyramid the better you end up.

    This is quite a specific sense, and isn't equal to just any kind of "unsustainable" system. Things can be unsustainable for reasons other than being a pyramid scheme relying on rapidly growing numbers of people being brought in by the ones already in the system.

    • Terr_  43 minutes ago

      (Going off topic, I know.) It's a less-egregious substitution, but Ponzi != Pyramid in some important ways. If they were the same, we wouldn't have needed to invent different terms. :)

          1. Actual Fraud
     * A Ponzi Scheme requires lying to people about what assets currently exist, or how much of the asset is reserved as "theirs". 
     * A Pyramid Scheme just asks people to collectively speculate (without guarantee) that tangible profits will arrive.
    2. Layers
     * A Ponzi Scheme can be one criminal and an indefinite number of unwitting victims. [0]
     * A Pyramid scheme (like the structure) requires *growing interior layers* of participants that blur the line between victimizer and victim.

      [0] Insofar as some Ponzi Scheme victims do not suffer losses, that's not because they were complicit, but because the criminal strategically chose to let some marks exit in order to maintain the illusion.