← Back to context

Comment by ben_w

4 hours ago

How many examples you need to get good.

Don't misunderstand: I'm happy saying AI models "think" or "have learned a thing", and for in-context learning I'd call them smart even by this definition…

…but also, any living creature that needed as many examples as machine learning currently needs, would starve to death before figuring out how to eat.

While training, machine learning processes (not just LLMs, also applies to e.g. self driving cars), are really really stupid and only make up for this by being really really stupid really really fast.

To what I wrote upthread: the "victories" of humanity over machine keep getting closer, but we have yet to wake up one day in great confusion as we find an entire city is no longer in communication with anyone, nor finding ourselves in a state of utter disbelief when the reports come in that the city stopped communicating because it is entirely gone.

Millions of years of evolutionary knowledge hard-coded into human systems, then it still takes 15+ years of us learning by example before we start to come online and be able to generalize solutions from a limited set of examples. I'm not sure this is as strong of an argument as you think it is. It also doesn't really matter when "we are trained differently" has no direct bearing on the end result.

  • We invented controlled fire perhaps a million years ago; at a generation gap of 25 years, that's 40,000 opportunities for evolution to pass on a mutation that does anything. Written language is around 210 generations old, the capacity to read and write isn't present in our nearest living relatives amongst the primates, and our various languages are wildly different to each other: the skill itself isn't evolved, though the capacity to learn the skill is.

    If humans learned like ML systems learn, (biblical) Methuselah would still have been failing the Sally-Anne test on his supposed deathbed at 969 years old, like some of the smaller early LLMs did.

    > It also doesn't really matter when "we are trained differently" has no direct bearing on the end result.

    The question was to ask for a definition such that AI could still count as "not smart" compared to humans. This fits.

    It's also why they're spiky intelligences, which I'm happily using right now to write code for me, but also do not trust in the slightest to identify the weeds in my garden. These submarines sure do swim fast*, but they're also very much disqualified for the Olympics.

    * https://en.wikiquote.org/wiki/Edsger_W._Dijkstra#1980s

If we're including the training process and not just the final product, why shouldn't we include the billions of years of natural selection encoded in DNA sequences?

  • We do.

    There's a lot of innate knowledge but all neuroscience demonstrates how incredibly flexible the brain is. Brains constantly learn and rewire.

    Here's a few things that I think show how crazy it is AND stress those points

      - people that have had corpus callosotomy (brain cut in half) *may* be indistinguishable from a normal person. Depends on how young you were when you underwent the procedure
        - true for most brain injuries
        - can even include the frontal cortex
      - you can learn to ecolocate
      - people with Aphantasia are indistinguishable from others
      - people without an internal monologue are indistinguishable from those with one
      - people can learn to use prosthetics
        - even without disabilities
        - or look into MRI scans with tool use
    

    You can convince yourself that we're just organic robots (after all, there's no magic), but you would be a fool to convince yourself we're the ordinary kind.

    We are constantly learning. You aren't just born with your knowledge and it stays static. We are extremely proficient at metalearning (learning how to learn, few shot learning, zero shot learning [0,1]). Our brains are constantly rewiring, able to heal from traumatic damage.

    I could go on and on. Does information pass down through genetics? Of course! But that's far from the whole story.

    I'm tired of people trying to make AI sentient by making humans robotic. Stop trying to trivialize everything and be okay not knowing the answer to everything. You're human, you're designed to learn and explore, not sit and argue from an armchair

    [0] and I mean these in the original sense. Not in the sense that you train on a billion examples of labeled animals and then congratulate yourself on your ImageNet-1k held out test performance. That's not zero shot, that's just a test set

    [1] I can literally make up words and you'll understand them. Or use words in novel ways. That's literally how slang works and how new words come to be. Don't be a walibanut ya glufus. Read some SciFi

  • Because our evolutionary environment doesn't contain cars, poetry, calculus, Star Craft, hamburgers, touch screen computers, or doors, and yet we are able to learn these things with (relative to a computer) very few examples.

    Most of the effort of evolution was making cells work at all, and even then it's a bit weird, e.g. no plant or animal produces vitamin B12 and we all get this from some bacteria and archaea.

    And evolution is kinda hard to time right: bacteria can reproduce in minutes, humans in decades, but only mutations that survive reproduction can be passed on. This makes it even starker as a difference: bacteria had order of 1e13 generations to become multicellular, while human DNA had about 40,000 generations to cope with fire, 220 generations for evolution to do anything with the invention of the wheel, and one generation to cope with the invention of Minecraft.

    The analogy here would be: DNA is to our brains like a VN replicator bootstrapping a computer all the way up to a bare-metal-no-OS untrained model, and perhaps a few crude "hard coded" modules like a smiling-face-detector. It's a lot, but it's also missing a lot. If biology used the models and training processes that are state of the art in ML, it would take around a millennia to talk like a child and still fail the Sally-Anne test, and million years or so to pass a degree.

    • I think you're underestimating how much knowledge about the world is encoded in human DNA, especially in the structure of the human brain at birth. It also depends how we count the "operations" used to train a human adult, even if we ignore the evolutionary history.

      I'm still going to deny the premise of your argument, becasue I think we should define intelligence in terms of capabilities. If a system can discover a cure for cancer or solve P vs. NP, it doesn't matter how many FLOPs it took to train.

      4 replies →