Comment by joebergeron
1 day ago
> Calling AI a collaborator isn't an attempt to anthropomorphize it. We can collaborate with a horse to travel
Naive question: What, practically speaking, is the delta between anthropomorphization of AI, which it seems like you are trying to avoid (fairly so!), and the sort-of animistic stance you seem more comfortable to apply?
I reckon that a lot of folks who might recoil at ascribing human qualities to LLMs would similarly feel discomfort comparing those models to a horse. While it's probably uncontroversial to say that horses are, coarsely speaking, "simpler" in many ways than humans, I suspect most people would agree that they are more complex than i.e., a "stochastic parrot".
Is the difference related to motive, maybe? That humans can have hidden preferences, and their actions can in fact betray their intended ends? That no matter how closely we observe someone, we may not glean any meaningful insight into their true motives? And we feel some (justified) discomfort in conferring this quality to AI? I think non-human animals can exhibit this sort of behavior too (albeit to a lesser degree). To me, it feels like restricting oneself to analogizing AI to a non-human animal in order to keep some distance from ascribing it consciousness may be less effective to that end than desired.
In place of your horse example, would it feel appropriate to liken AI to, say, a compiler peppered with bugs? Program - prompt - goes in, executable - agent actions - come out; mostly what we expect, but occasionally goes off the rails in ways we cannot understand. The caveat there, I suppose, is that we have not until now created a compiler which we cannot sensibly decompile.
Fun to think about.
Scary as well. You can control a tool, but not necessarily a sufficiently intelligent collaborator. If you approach AI as a collaborator, the challenge isn't to understand it and predict it, as OpenAI failed to do with their own product, but to motivate it correctly and and constrain it sufficiently to guarantee safety. You need guardrails for when you get the alignment wrong. How strong can guardrails be made though?
The public has this sci-fi idea that AI can be constrained with simple, Asimov style laws. The problem is that LLM's don't think in the way humans do. They can produce results of dizzying complexity but hallucinate or fail to do basic arithmetic. You can't currently rely on AI to follow simple rules like "Don't access an outside network.". This is a problem that needs to be addressed urgently.