Comment by ben_w
5 days ago
IMO, hard part is likely the lack of haptic feedback, not that it's just a simple gripper. Small and big feedback, so you don't break eggs (not accidentally at least), and also don't pick up a heavy load that makes the robot topple over and give the remote operator motion sickness in the process.
There's a surprising number of labour simulators sold as VR headset games:
https://skyhookgames.com/lawn-mowing-simulator-vr-is-out-now...
I’m not sure remote operations needs to be like a VR sim or joystick control of every movement. It might be something more similar to prompts. “Go to front door” “bring in packages” “open packages” “sort trash into pile” <click for item select>”is not trash” “take trash to bin”. Once the house is mapped and a good deal of these routines are learned they can be executed with some fuzzy input to the automation. In my mind this is still full teleoperation
It would be great if the operator could say things like, in above example, “you forgot to close the door, make sure you never leave this door open, even if you have to open it for a task and it’s not explicitly said you should always imply the door needs to be closed after the task. As a general rule, the door should never be open longer than 1 minute longer than it needs to be for task performance” or something similar and this trains the model over time.