Comment by mlpoknbji
13 days ago
My favorite early chatgpt math problem was "prove there exists infinitely many even primes" . Easy! Take a finite set of even primes, multiply them and add one to get a number with a new even prime factor.
Of course, it's gotten a bit better than this.
IIRC, that is actually the standard proof that there are infinitely many primes[1] or maybe this variation on it[2].
[1]: https://en.wikipedia.org/wiki/Euclid%27s_theorem#Euclid's_pr...
[2]: https://en.wikipedia.org/wiki/Euclid%27s_theorem#Proof_using...
Yes this is the standard proof of infinitely many primes but note that my prompt asked for infinitely many even primes. The point is that GPT would take the correct proof and insert "even" at sensible places to get something that looks like a proof but is totally wrong.
Of course it's much better now, but with more pressure to prove something hard the models still just insert nonsense steps.