Comment by x3haloed

The readme on Github does say the system produces hallucinated sources https://github.com/anonnerdfrenzy/survival-llm#how-the-model..., but really you're correct, for this kind of highly targeted use case the LLM's summary could probably be ignored entirely and the system could work extremely well even just as a search engine: index all of the content up front, chunked by paragraph and maybe enriched with a HyDE step, have the LLM facilitate running a handful of hybrid keyword/semantic searches based on the user's question and reranking the results, and show the exact pieces of text in the corpus that are most responsive to their question. No summary needed.

A great strength of LLMs is how they provide useful answers even if the prompt has glaring spelling or grammatical mistakes. If your prompt is not great, as you said, they will reply in a way that helps you identify that flaws, and learn how to steer the model better.

They're amazingly useful tools, but they're nowhere the level believers want them to be, not yet.

The summarization is the main issue here. A fuzzy search is far less controversial. The LLM companies have done a good job synonymizing "AI" with their products however.

In a true emergency, there are often many cases where you need not just information retrieval, but "thinking and reasoning" to deal with unprecedented situations, right? Is it technically possible to balance lightweight local performance with thinking capability, while avoiding the risk of hallucinations in life-or-death instructions?

Can a bad LM implementation surface bad references? Yes. Are LMs overwhelmingly useful for research? Yes.

But at this point why even bring a full model into scope in the first place.. just run some cheap embedding retriever

Nevermind hallucinations, the donkey-work is important; why bypass the hours of research, which trains our in-built LLM and can lead to new realizations, and get the carrot without much effort? This LLM stuff feels like a candybar; the anticipation of the magic feels good, the thing tastes good while you're eating it, and everything thereafter is shit, but we easily forget, and reach for the next candybar. I appreciated Blindboy Boatclub's take in his podcast episode "Artificial intelligence is Disgusting and it will never replace Artists" from 2026 March 17. I also appreciate Electrical Engineer & Computer Scientist Timnit Gebru's perspective on these stochastic parrots. While intraspecies diversity confers, up to a point, a survival benefit, consider the long-term consequences of playing this particular game. We don't have to eat every candybar the rich mostly-guys ply us with.