Comment by amelius
2 months ago
Studies of LLMs belong in their own field of science, just like psychology is not being studied in the physics department.
2 months ago
Studies of LLMs belong in their own field of science, just like psychology is not being studied in the physics department.
Interestingly enough, for a while physics used to be studied by philosophers (and used to be put in the natural philosophy basket, together with biology and most other hard sciences).
¸That field is called Machine Learning.
No that's still like putting cellular biology and psychology in the same bin.
The intersection of physics isnt psychology it is philosophy, and the same is true (at present) with LLM's
Much as Diogenes mocked Platos definition of a man with a plucked chicken, LLM's revealed what "real" ai would require: contigous learning. That isnt to diminish the power of LLM's (the are useful) but that limitation is a fairly hard one to over come if true AGI is your goal.
Is it because we haven't invented something better than backpropagation yet?
From what I understand, a living neural network learns several orders of magnitude more efficiently than an artificial one.
I'm not sure where that difference comes from. But my brain probably isn't doing back propagation, it's probably doing something very different.
Your brain is doing several different things, because there are different parts of your brain.
(eg different kinds of learning for long-term memory, short-term memory, languages, faces and reflexes.)
What is "contigous" learning, and why is it a hard requirement of AGI?
What do you mean by the intersection of physics?
The intersection of what with physics?
The intersection of disciplines.
Sir Roger Penrose, on quantum consciousness (and there is some regret on his part here) -- OR -- Jacob Barandes for a much more current thinking on this sort of intersectional exploratory thinking.
That is a very interesting thought!