Comment by isoprophlex
3 months ago
People agreeing and disagreeing about the central thesis of the article, which is fine because i enjoy the discussion...
no matter where you stand in the specific o1/o3 discussion the concept of "question entropy" is very enlightening.
what is the question of theoretical minimum complexity that still solves your question adequately? or for a specific model, are its users capable of supplying the minimum required intellectual complexity the model needs?
Would be interesting to quantify these two and see if our models are close to converging on certain task domains.
Good stuff.
I am going to try to start measuring my prompts.
Thinking about it, I am not sure what the entropy is for the above vs "start measuring prompts".
That's a tough one, I'm not sure how to get a quantifyable number on it except for painstakingly ablating a prompt until the answer you get becomes significantly degraded.
But then still, how do you measure how much you have ablated your prompt? How do you measure objectively how badly the answer has degraded?