← Back to context

Comment by viraptor

11 hours ago

It's an interesting review, but it seems to mix some things together. Gruber's claim is that we shouldn't sacrifice quality, but none of those sources show that's what's happening. Having less diversity in answers could mean you're more consistently getting a better/worse/same quality answer. There are lots of people complaining about not being able to get a real temperature=0 answer - they'd actually welcome the consistency too.

The practical example of not liking the sentence is not great in my opinion. Instead of just re-rolling the answer, why not continue in that thread and say what you don't like? Or request 20 alternatives, say which go in the right direction and request 20 more? Retrying from scratch with one sample is a waste of time and doesn't indicate anything about quality.

Basically, it's all theoretical - show us the actual benchmarks/samples where the quality is consistently degraded.