Comment by veunes
2 days ago
This is probably part of why machines are doing so well at counterexamples. They have no aesthetic commitment to the conjecture and no embarrassment about producing something ugly
2 days ago
This is probably part of why machines are doing so well at counterexamples. They have no aesthetic commitment to the conjecture and no embarrassment about producing something ugly
That's not why.
It's because counterexamples are easy compared to proofs which require new mathematics.
GenAI is great at combining existing things in new ways (interpolation). It's terrible at creating new things from scratch (extrapolation).
This is the wrong way to think about mathematical (or any other kind of) creativity in my opinion. In the extremely high-dimensional space of "ideas" (whatever that means) there are almost certainly profound ideas that are the interpolation of existing knowledge, i.e. the curse of dimensionality. It's not at all clear that you need to extrapolate from existing knowledge to be creative.
I really don't think that's true. Of course after the fact a counter example looks easy. Because look at it, it's obviously that it doesn't work. But that disregards the process of finding it. That requires great creativity(or computational resources if the problem is tractable at all).
>GenAI is great at combining existing things in new ways (interpolation). It's terrible at creating new things from scratch (extrapolation).
I think you believe a fallacy about how human cognition works if you think we actually do something different than interpolation
Then where do new ideas come from? Do the muses bestow divine creativity upon artists, poets, and musicians?
1 reply →
I think this is somehow related to GenAI’s issue of “confident incorrectness”. I’ve been trying to prompt GenAI with some highly challenging prompts recently, especially terse ones:
> Which Anjunabeats and Anjunadeep compilations have the most breakbeat tracks?
GenAI was able to give me breakbeat tracks that were on the Anjuna labels, but it completely flubbed on whether and which those tracks were on compilations. It was very confidently, mostly incorrect.
I think this is somehow isomorphic to the “interpolation vs extrapolation” issue. GenAI is forced to generate an answer, and there’s no mechanism by which we can interrogate the model for its confidence (at least, that’s my understanding).
And yet, I asked “How confident are you in your answer along different dimensions, ie whether those tracks are breakbeat, whether they are Anjuna tracks, or whether they are on those compilations?”
And its answer was surprisingly satisfying, it only gave 30% confidence that the tracks were on the compilations, which is where it flubbed.
Anyway, personal observations, GenAI is meant to be interacted with, not just a single prompt black box.
4 replies →
I don’t think that’s true. Lots of mathematicians excel at (and revel in) finding weird counterexamples and love the strangeness of it all. John H Conway being my favourite- Look up the Conway knot[1] or the Conway base 13 function[2] for famous examples.
Ugliness is in the eye of the beholder. Lots of counterexamples are very beautiful. For example, the Dirichlet function (f(x) = 1 if x is rational, 0 otherwise) is a source of very beautiful counterexamples. Eg it is discontinuous everywhere but f restricted to only rational numbers is continuous on all rationals and likewise f restricted to irrationals is continuous on the set of irrationals (R-Q).
[1] The Conway knot has 11 crossings yet shares the same Alexander polynomial as the “unknot” which has no crossings at all. It took 50 years to decide the question of whether it has a basic property known as “sliceness” https://en.wikipedia.org/wiki/Conway_knot
[2] Conway’s base 13 function Was invented as a counterexample to the converse of the intermediate value theorem. That is, it satisfies the intermediate value property while being everywhere discontinuous (which breaks my brain completely) https://digitalresearch.bsu.edu/mathexchange/wp-content/uplo...
This reminds me of the Go Grandmaster speaking out after losing to AlphaGo, that the model has no sense of "aesthetic play", as long as it would lead to a win within the rules.
I thought there was a lot of buzz about AI creativity after the infamous move 37 in that series?
Yes but it was so shocking because it was such an inhuman, "unaesthetic" play. It was considered to be "creative" in that no human would have thought of making the move, so it can't simply be copying human play.
1 reply →
if humans could do it, it would be called beautiful. defining something to be ugly is cope
They're trained on human data. I would expect them to emulate human biases as closely as possible.
This is where harness, and the fact that a machine can be endlessly prompted to try again comes in.
Even if an LLM starts by pursuing things that follow human bias, continuous failures and re-prompting to try something different will eventually force it to consider things outside of what ever biases it has.
You can do the same thing to a human. But most people would consider it unethical to lock someone in a box and force them to keep trying to solve the same problem over and over again until they figure it out.
Your comment stopped me in my tracks a little bit.
Is a 'bias' in a piece of writing generally a property of word to word choice and sentence to sentence construction or is it something more nebulous? Especially in terms of the appreciation of mathematics and someone's hesitance about publishing a mathematical argument they think is ugly or brute forced in some way.
> Is a 'bias' in a piece of writing generally a property of word to word choice and sentence to sentence construction or is it something more nebulous?
You might be fascinated when you read the story of Golden Gate Claude: https://www.anthropic.com/news/golden-gate-claude
Is it? I'd expect most of the training set to be synthetic data extrapolated from a small set of human authored texts.
Most of the training set is half of the Internet. LLMs are pre-trained on general set of human biases and patterns of thinking.
It's more complex than that, especially as post training is often goal based.
I wouldn't have expected that there was post training specifically on the issue of looking for proofs vs counter examples. But it might be that other post training has a side effect of making AIs better at looking for counter examples. I wonder if these agents are overall less biased and more rational than humans. Can you expand on what you mean by goal based training?
No, they are not. Specifically, this was a method based specifically on learning from scratch, like most modern AI models.
Why do you think it's called Alpha ZERO?
I think you're in the wrong thread. This one is about Claude producing counterexamples. The Go thread is next door.
Programmers love to give things names that they think sound cool
1 reply →