I did some tests with heavily math oriented programming using ChatGPT and Gemini to rubber-duck (not agentic), going over C performance tuning, checking C code for possible optimizations, going over math oriented code and number theory, and working on optimizing threading, memory throughput, etc. to make the thing go faster, then benchmarking runs of the updated code. Gemini was by far better than ChatGPT in this domain. I was able to test changes by benchmarking. For my use case it was night and day, Gemini's advice generally quite strong and was useful to significantly improve benchmarked performance, ChatGPT was far less useful for this use case. What will work for you will depend on your use case, how well your prompting is tuned to the system you're using, and who knows what other factors, but I have a lot of benchmarks that are clear evidence of the opposite of your experience.
Which models? It's completely uninformative to say you compared "ChatGPT" and "Gemini." Those are both just brand names under which several different models are offered, ranging from slow-witted to scary-smart.
I'm not OP but I tried to ask it today to explain to me how proxies work, and it felt like it couldn't give me an answer that it couldn't attribute to a link. While that sounds good on paper, the problem is that no matter what you ask, you end up with very similar answers because it has less freedom to generate text. While on the other hand, even if attributes everything to a link, there is no guarantee that link actually says what the LLM is telling you it says, so it's just the worse of both worlds.
ChatGPT on the other hand was able to reformulate the explanation until I understood the part I was struggling with, namely what prevents a proxy from simply passing its own public key as the website's public key. It did so without citing anything.
I did some tests with heavily math oriented programming using ChatGPT and Gemini to rubber-duck (not agentic), going over C performance tuning, checking C code for possible optimizations, going over math oriented code and number theory, and working on optimizing threading, memory throughput, etc. to make the thing go faster, then benchmarking runs of the updated code. Gemini was by far better than ChatGPT in this domain. I was able to test changes by benchmarking. For my use case it was night and day, Gemini's advice generally quite strong and was useful to significantly improve benchmarked performance, ChatGPT was far less useful for this use case. What will work for you will depend on your use case, how well your prompting is tuned to the system you're using, and who knows what other factors, but I have a lot of benchmarks that are clear evidence of the opposite of your experience.
Which models? It's completely uninformative to say you compared "ChatGPT" and "Gemini." Those are both just brand names under which several different models are offered, ranging from slow-witted to scary-smart.
why?
I'm not OP but I tried to ask it today to explain to me how proxies work, and it felt like it couldn't give me an answer that it couldn't attribute to a link. While that sounds good on paper, the problem is that no matter what you ask, you end up with very similar answers because it has less freedom to generate text. While on the other hand, even if attributes everything to a link, there is no guarantee that link actually says what the LLM is telling you it says, so it's just the worse of both worlds.
ChatGPT on the other hand was able to reformulate the explanation until I understood the part I was struggling with, namely what prevents a proxy from simply passing its own public key as the website's public key. It did so without citing anything.
did it tell you how smart your questions were, too?
Maybe this week it is.