Comment by olalonde
3 hours ago
> Decision models have the potential to have an even larger impact on the Real World than LLMs have to this point
Why?
3 hours ago
> Decision models have the potential to have an even larger impact on the Real World than LLMs have to this point
Why?
A major limitation of directly using "System 1 decision models" (a.k.a., logistic regression, and related uncalibrated classifiers over the output/logit space) in enterprise settings (or other high-stakes settings) is that such estimators are not reliable estimators of the predictive uncertainty in the presence of covariate shifts, and such estimators also lack a means of instance-wise data attribution (i.e., interpretability-by-exemplar), so they're not the ideal estimator for verification, routing, uncertainty over retrieval and tool-calls, etc.
For decision-making with neural networks, we instead need the older idea of estimators of the predictive uncertainty with constraints in the feature-representation space (over training/support of the estimator), as with Similarity-Distance-Magnitude estimators: https://pypi.org/project/reexpress-sdm/
If youre the one or have spoken to someone implementing "AI solutions" inside large companies recently, a decent chunk of it is soft policy enforcement with very basic context. They moved from gemini 2.5 flash lite type models to jev. Which is also why I found the price comparisons to "GPT Astra" on Twitter rather funny.