Comment by jatora

Most recent problems were related to topology, but it can take the wrong direction on many things. This is not an LLM fault; it's a training data issue. If historically a given direction of inquiry is favored, you can't fault an LLM for being biased toward it. However, if small volume and recent results indicate that path is a dead end, you don't want to be stuck in fruitless loops that prevent you from exploring other avenues.

The problem is if you're interdisciplinary, translating something from one field to one typically considered quite distant, you may not always be aware of historic context that is about to fuck you. Not without deeper insight into what the LLM is choosing to do or read and your ability to infer how expected the behavior you're about to see is.