Comment by addaon
4 months ago
“Up to X” produces a relatively strong bias for producing X yesses. “For each of these possible labels, write a sentence describing whether it applies or not, then summarize with the word Yes or No” does a bounded amount of thinking per label and removes the bias, at the cost of using more tokens (in your pre-processing phase) and requiring a bit of post-processing.
Those are just simple prompt examples: obviously more prompt engineering would be necessary.
However, modern LLMs, even the cheaper ones, do handle the up to X constraint correctly without consistently giving X.