Comment by jameshart
2 days ago
It’s easy for people to understand that if they point the powerdrill into a wall the failure modes might include drilling through a pipe or a wire, or that the powerdrill should not be used for food preparation or dentistry.
People, in general, have no such physical instincts for how using computer programs can go wrong.
Which is in part why rejection of anthropomorphic metaphors is a mistake this time. Treating LLM agents as gullible but extremely efficient idiot savants on a chip, gives pretty good intuition for the failure modes.