Comment by loveparade
2 hours ago
I'm a lot more skeptical. I'm not a mathematician, but from my use of LLMs a very clear pattern of what they are good and bad at has emerged. They are extremely good at combining large amounts of information, and it seems this is what the current AI results in mathematics are. There are so many subfieleds of math with ties to each other, so many papers and niche results, that no human could ever read, comprehend, connect and organize that information in their brains. Pretty much all of it was created by humans. And there is real value in doing this and creating new results from what we've already discovered.
But there also is another type of discovery that requires taking a step back and looking at the problem from a different angle. If you are an engineer, how often has an LLM told you (without you explicitly prompting for it): Wait, what you are doing here doesn't really make sense, there exists a much more elegant abstraction that nobody has thought of, let's remove all that code, let's tackle the problem in a different way by thinking from first principles. Pretty much never. But in science a lot of the biggest discoveries have come from this kind of first principle thinking, questioning existing work and approaches and going against what already exists, not combining all existing data which is likely to be just a local optimum.
LLMs are inexplicably good at working within any tight feedback loop to coerce the desired solution. This is precisely why proof assistants + LLMs are non intuitively successful.
This is also why they're so good at creating three.js or Blender work when the output is so easily constrained to "Look exactly like that". I recently posted https://www.ambionix.com/blog/introducing-the-czp-1/ on here, and the audio engine in that was developed in that way.
It is true that it would be astounding to find if anyone has seen a LLM produce any useful generalization of anything resulting in a simplification. They seem to have a direct tendency to do the opposite. The brutal reality is humans have also undervalued this capability for a long time (I think the Poincare/Hilbert debate is relevant) to the point we are also taught that generalizations are, generally, bad and wrong.
As a non-mathematician, I've been wondering if LLMs will be able to leverage their knowledge across all domains to help build a "simplified/unified" version of math.
Like, I think there have been attempts at this across the field. (I could be wrong!) But it requires a lot of labor and a lot of cross domain knowledge to complete. Both things that AI have.
I was working on a project recently where I wanted to express a relationship (that I knew existed, but didn't know how to express) between four measured scalar values. Astra insisted there was no relationship, and that any correlation wouldn't make sense.
Eventually, by walking through them, it proposed an additional fifth value and from there was able to tie everything together.
Sometimes you just gotta hit the machine until it works again.
Your experience mirrors so many managers' experience with engineering teams...
It's an unfortunate truth that there is the right way to do things, and then there is the way they have to be...
LLM's are very outcome oriented and i think this is where your observation comes from. You tell LLM you want something, it doesn't even question the premise and just starts calculating 100 different ways to get there. LLM's have knowledge but lack wisdom.
Have you ever asked it to?
> there exists a much more elegant abstraction that nobody has thought of, let's remove all that code, let's tackle the problem in a different way by thinking from first principles.
You literally just have to ask it. Before I left software engineering in April, I was using Claude for re-architecture all the time.
But no, it doesn't assume it should re-architect what you're handing it when you haven't asked it to.
That's the point. When a human works on a problem they realize themselves "wait, i probably should re-architect now" - of course you can ask an LLM to do that, but at that point you already know yourself what you need, which defeats the point of LLM working on difficult problems that require insight automatically. A lot of these math problems are sessions over many hours. And of course you can also ask "Think about whether to re-architect at each step" and it will never do the right thing because the context it builds up for itself drives it into a specific solution space. It's literally trained to complete exactly that.
why did you leave?
> You literally just have to ask it.
why doesnt it ask itself before proceeding?
Because if you asked it to fix a bug and every time it responds with paragraphs of how you could re-architect the system, it would be incredibly annoying.
1 reply →
> If you are an engineer, how often has an LLM told you (without you explicitly prompting for it): Wait, what you are doing here doesn't really make sense, there exists a much more elegant abstraction that nobody has thought of, let's remove all that code, let's tackle the problem in a different way by thinking from first principles.
I explicitly request it. It's not great at coming up with interesting ideas, but neither am I, and it can sure iterate on them faster than I can...
yes and this is the difference between human intelligence and the massive raw dumb intelligence of the computah