Comment by sim04ful

I reached a similar conclusion: LLMs should only really sit at the terminals of request fulfilment.

1. User request understanding: natural language -> a more rigorous representation, in my case Datalog.

2. Result interpretation: facts and derived facts -> natural language.

Between those terminals, the work should be mechanical reasoning over some ontology or formal knowledge structure.

That connects to another principle I've been thinking about, which I call Weathering: useful reasoning should change the shape of the system. If an LLM has already had to infer a relation, mapping, rule, or abstraction, repeated use should wear that inference into the system so that the next similar request doesn't require discovering it again from scratch.

With continued use, a weathering-capable system should therefore require less and less probabilistic intelligence for recurring work. Put another way, there should be a declining marginal cost of cognition since the products of intelligence harden into structure that can subsequently be reused and evaluated mechanically.