← Back to context Comment by satvikpendem 1 day ago Can't I just parallelize my LLM calls myself for each question? 5 comments satvikpendem Reply orbital-decay 1 day ago You can. It will be expensive, slow, and less reliable than a specialized model. zwily 1 day ago Anything you can do in Jev can be done with an LLM at much greater cost and latency. mmnfrdmcx 21 hours ago Agree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs. esafak 20 hours ago That number is a big deal, assuming it is well calibrated. Did they talk about calibration? 1 reply →
orbital-decay 1 day ago You can. It will be expensive, slow, and less reliable than a specialized model.
zwily 1 day ago Anything you can do in Jev can be done with an LLM at much greater cost and latency. mmnfrdmcx 21 hours ago Agree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs. esafak 20 hours ago That number is a big deal, assuming it is well calibrated. Did they talk about calibration? 1 reply →
mmnfrdmcx 21 hours ago Agree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs. esafak 20 hours ago That number is a big deal, assuming it is well calibrated. Did they talk about calibration? 1 reply →
esafak 20 hours ago That number is a big deal, assuming it is well calibrated. Did they talk about calibration? 1 reply →
You can. It will be expensive, slow, and less reliable than a specialized model.
Anything you can do in Jev can be done with an LLM at much greater cost and latency.
Agree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs.
That number is a big deal, assuming it is well calibrated. Did they talk about calibration?
1 reply →