Comment by dgellow
15 hours ago
In VC investing there is the saying that most of the value comes from the outliers. I think it’s the same for a lot of domains. I read an interesting article recently on LLMs homogeneity when writing fiction: https://arxiv.org/abs/2604.03136
It seem that at scale LLMs output is the average of their dataset, they all cluster around the same space, where human creativity comes with more variance, exploring way more of the space
Can't the LLMs build scripts/tools etc that help generate creative ideas?
e.g. have the LLM generate multiple lists of characters, themes etc and then have a script hooked up to a RNG pick from the lists to create more "creative" ideas.
I think there’s merit to this approach, particularly to highly parallelizable tasks.
Rather than giving many agents the same prompt, introduce random variations that lead each agent in different directions. For a single bug, you might fire three agents, and later select the best result:
“Fix this bug. The solution is a trivial typo.”
“Fix this bug. The solution centers on correcting a bad assumption.”
“Fix this bug. The solution will require a complete redesign.”
You could follow the same idea with varying the input context, or by adding artificial constraints to the solution. Like telling each agent to “fix the bug, by only modifying file a/b/c”
That's pretty low-value creativity. Real innovation doesn't come from mad libs.
I would highly recommend the "Everything is a Remix" series on YouTube to show how a lot of "creative" work is really just combining different items.
https://www.youtube.com/watch?v=nJPERZDfyWc
2 replies →
This is roughly the same as putting a group of random people and telling them to think outside the box which in my experiences never yields much of thinking outside of the box
> This is roughly the same as putting a group of random people
In the book Range by David Epstein [0], he mentions that research has shown that groups of people with very mixed backgrounds leads to much faster rates of problem solving and innovation. The reasoning is that someone will have an experience or piece of knowledge that directly applies to the problem.
As an example, two different groups were given the same problem to solve. One was a homogeneous group of academics. The other had a mixture of academics and non-academics. Both had to solve a problem that involved flowing liquid. The mixed group solved it faster due to one of them having had a father who was a plumber.
0 - https://amzn.to/4wWd50y
3 replies →
All the LLM needs is a tiny rate of variance then you can scale it and it will out produce humans.
That’s already what we currently have. Look for “temperature” in the context of LLMs. They are already tuned to have variance.