Comment by satvikpendem 1 day ago Can't I just parallelize my LLM calls myself for each question? 5 comments satvikpendem Reply orbital-decay 1 day ago You can. It will be expensive, slow, and less reliable than a specialized model. zwily 1 day ago Anything you can do in Jev can be done with an LLM at much greater cost and latency. mmnfrdmcx 1 day ago Agree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs. esafak 21 hours ago That number is a big deal, assuming it is well calibrated. Did they talk about calibration? 1 reply →
orbital-decay 1 day ago You can. It will be expensive, slow, and less reliable than a specialized model.
zwily 1 day ago Anything you can do in Jev can be done with an LLM at much greater cost and latency. mmnfrdmcx 1 day ago Agree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs. esafak 21 hours ago That number is a big deal, assuming it is well calibrated. Did they talk about calibration? 1 reply →
mmnfrdmcx 1 day ago Agree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs. esafak 21 hours ago That number is a big deal, assuming it is well calibrated. Did they talk about calibration? 1 reply →
esafak 21 hours ago That number is a big deal, assuming it is well calibrated. Did they talk about calibration? 1 reply →
You can. It will be expensive, slow, and less reliable than a specialized model.
Anything you can do in Jev can be done with an LLM at much greater cost and latency.
Agree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs.
That number is a big deal, assuming it is well calibrated. Did they talk about calibration?
1 reply →