Comment by rapjr9
2 days ago
Dexterity is also hard because, at least in humans, it relies on knowing something of the nature of an object _before_ manipulating it. Is it light or heavy? Soft or rigid? Is it a bag of popcorn, popcorn kernels, a bag of powder, or a pillow? How tightly is it packed in the bag? Fabric or cardboard? Attached to other objects or not? Is the USB plug the right type and oriented correctly? (Even humans have trouble with this one.) Does it have a slippery surface or a grippy surface? To be immediately successful in manipulation, pre-knowledge based on sensing and identification is usually required. Possibly it would be ok if a robot took several tries to figure this out based on some general principles, but it will seem clumsy and be slower. It seems there is an ontology problem here, which requires understanding a lot about the world in order to be able to successfully manipulate it.
More generally, continuous learning in real-time is something current models don't do well. Retraining an entire LLM every time something new is encountered is not scalable. Temporary learning does not easily transfer to long term knowledge. Continuous learning still seems in its infancy.
Also when we don't know the properties of an object we are about to manipulate we'll approach it cautiously and learn it before we apply too much force. This tends to happen transparently and quickly for adults, but for infants you can watch it play out more slowly.
My guess is that it helps a lot that we have flexible cushioned fingertips that are highly sensitive to pressure. That's a hardware feature that robots mostly lack.
Our evolution of nails, as opposed to the much more common claw, is another part of the symphony of touch. It provides a hard backstop to assist with touch sensitivity.
Imagine if the tip of your finger could just bend back. It would be way harder to know what you’re touching!
2 replies →