Comment by shybear
8 hours ago
It seems like alot of scientific advancements occurred by someone applying technique X from one field to problem Y in another. I feel like LLMs are much better at making these types of connections than humans because they 1) know about many more theories/approaches than a single human can 2) don't need to worry about looking silly in front of their peers.
Exactly. Much of the intellectual work is, in fact, intellectual labor. It’s mostly about combining various information in one place — the exact task that LLM far outperforms human. People traditionally misclassified this class of work as “creative”. It’s not really.
Having a new insight that leads to the combination of two distinct ideas is definitionally creative.
You can say this problem needed a low amount of total creativity, but saying it's void of all creativity seems wrong.
Creative bit is figuring out two or more bits that might work together for something new. Labour part is combining that especially if it is actually laborious.
Which get to other possibility of having list of distinct things and then iterating over all pairs or combinations. Which I probably would not qualify as "creative" work.
Yeah, I've been grappling with the definition of creativity too. There's a gamedev talk [0] on creativity that gave me useful perspective. Here's what I wrote elsewhere:
---
i've been thinking about raph's definition of creativity [0]: permuting one set of ideas with another set of ideas
(or trying an idea in new contexts)
this is a systematic process, doable even by machine once enough pattern libraries have been catalogued.
on a small scale, there's sprint.cards [1] or oblique strats [2]. on a large scale, there's llms...
it's freeing to approach creativity as a deliberate practice rather than waiting on some fickle muse. yet it's a bit disappointing to see idea generation so mechanical and dehumanized.
i am comforted by the value of mushy human abilities surrounding the creative process:
mostly 1) taste, the ability to recognize pleasing output,
...
[0] https://www.youtube.com/watch?v=zyVTxGpEO30
[1] https://sprint.cards/
[2] https://stoney.sb.org/eno/oblique.html
I agree except: this is creative work. Creativity can be and is being mechanised. True originality is extremely rare. Most novelty is the repurposing of one idea or concept elsewhere in a way we call find surprising, but the choice to apply A to B could have been made for any reason including mechanical: very many inventions are accidents. In-depth knowledge / conceptual understanding of something is built on abstraction, and abstractions are portable.
If you had a list of N concepts and M ways to apply them you could try all N*M combinations, and get some very interesting results. For a real example, see the theory of inventive problem solving (TRIZ)'s amusing "40 principles of invention" by Soviet inventor Genrich Altshuller. https://en.wikipedia.org/wiki/TRIZ
What is your idea of "creative"/"creativity" then?
Coming up with said novel techniques in the first place. Arguably something that most humans can't really do reliably or at all.
2 replies →
Maybe all intellectual work is intellectual labor?
This is exactly what creativity is.
> Much of the intellectual work is, in fact, intellectual labor.
That's a great point. It's in line with research being carried on the backs of graduate students, whose work is to hyperfocus on areas.
Isn't that science too?
> Much of the intellectual work is, in fact, intellectual labor.
Not surprisimg, because the two words you used are synonyms. Who did ever classify mathematical work as creative? Kids in third grade math class?
> that LLM far outperforms human.
LLMs only outperform humans in creating loads of bullshit. 6 years in and they remain shiny toys for easily impressionable idiots.
As I understand it, models form connections (weak or strong) between everything in their training sets, even the smallest details. They've already made other breakthroughs directly because of this ability and this line of research is likely to be incredibly fruitful.
> someone applying technique X from one field to problem Y in another
Witten is the canonical example of someone taking mathematics techniques and applying them to physics problems, but what made him legendary was the opposite direction: he used physical intuition and string theory to solve open problems in pure mathematics.
This is what I personally consider as "reasoning" ... knowledge generalization and application across domains.
Less reasoning than a dimension of brute force unfamiliar to human brains.
Trying to diminish this as brute force (something by the way that is categorically not 'unfamiliar to human brains' - as anyone who has every worked on complex slippery problems will tell you) is foolish, when the models hypothesize along the way to their solutions. That's reasoning.
Familiar but isn't effective enough for surviving.
This is what I have been doing. I don't think I've made any amazing breakthroughs, but at the same time I can't help but feel like I've come across some white paper-worthy realizations. Being able to correlate across a lot of domains I feel like I intuitively understand but have no depth of knowledge has been a fun exercise in LLM experimentation.
> It seems like alot of scientific advancements occurred by someone applying technique X from one field to problem Y in another.
Yeah, you should look into the Langlands project sometime
I'm thinking once we have much of the math literature formalized it's going to be possible to mine commonalities like that. Think of it as automated refactoring, applied to math.
As a civilization we went the left-brained/sequential/language based way of thinking (with computers and AI being the crown achievement of it). Personally i for example remember like around 3rd grade i switched from the whole-page-at-once reading mode into the word by word line by line mode and that mode stuck with me since then (at some point while at the University i had for some period of time, probably it was the peak of my abilities, some more deep/wide/non-linear perception into at least my area of math specialization, though not sure whether it was a mastery by the left brain or the right brain got plugged in too) LLMs will definitely beat us in that sequential way of thinking. That makes me wonder whether we will have to push into our whatever is still left there right-brainness, and whether AI will get there faster too. May be we'll abandon the left-brain completely leaving it to AI.
If that is your hope you are probably in for a rude awakening. Left brained/right brained is a wooden exaggeration according to more recent research [1].
[1] e.g. https://www.sciencenewstoday.org/left-brain-vs-right-brain-t...
Well, maybe. The poster you replied to wasn't discussing literal neuroanatomy, they were using "left/right-brained" in the colloquial, metaphorical sense.
accuracy and creativity are often quite difficult to achieve at the same time. Looks like LLM can do it, even though one can question how creative it really is...
Can one? It's surpassed the creativity of humans in this one problem at least.
[dead]