← Back to context

Comment by paulddraper

3 hours ago

"LLMs just interpolate their training data"

Cracks me up.

What exactly do we think that human brains do?

10 comments

paulddraper

Reply
charlie90  2 hours ago

I agree. Humans are given a body that lets them "discover" things on accident, test out ideas, i.e. randomness.

As in, I would hazard a guess the discovery of the wheel wasn't "pure intelligence", it was humans accidentally viewing a rock roll down a hill and getting an idea.

If we give AI a "body", it will become as creative as humans are.

omnimus  3 hours ago

That has been the question since the beginning of humans.

Maybe computers can help understand better because by now it's pretty clear brains aren't just LLMs.

  • baq  3 hours ago

    The optimists believe brains are very special and we’re far from replicating what they do in silicon.

    The pessimists just see a 20W meat computer.

__s  1 hour ago

Creativity is hard. Pretty much needs a fuzzer process to generate new strings, mostly nonsense, & pick up when that nonsense happens to be correct

ActorNightly  2 hours ago

I love this comment because it so clearly highlights the difference between intelligence and reasoning.

A lot of people across all fields seem to operate in a mode of information lookup as intelligence. They have the memory of solving particular problems, and when faced with a new problem, they basically do a "nearest search" in their brain to find the most similar problem, and apply the same principles to it.

While that works for a large number of tasks this intelligence is not the same as reasoning.

Reasoning is the ability to discover new information that you haven't seen before (i.e growing a new branch on the knowledge tree instead of interpolating).

Think of it like filling a space on the floor of arbitrary shape with smaller arbitrary shapes, trying to fill as much space as possible.

With interpolation, your smaller shapes are medium size, each with a non rectangular shape. You may have a large library of them, but in the end, there are just certain floor spaces that you won't be able to fill fully.

Reasoning on the flip side is having access to very fine shape, and knowing the procedure of how to stack shapes depending on what shapes are next to it and whether you are on a boundary of the floor space or not. Using these rules, you can fill pretty much any floor space fully.