Comment by hahahaa
17 hours ago
Oh yeah I mean how do the phones do that. I guess it uses dead reckoning with visual cues and some machine learning.
17 hours ago
Oh yeah I mean how do the phones do that. I guess it uses dead reckoning with visual cues and some machine learning.
"SLAM" is the term to google.
on iPhone 12 Pro Max and later (maybe only on Pros, I forget) they have a LiDAR sensor built in, allowing for impressively accurate 3D scans of objects with a free app like Scaniverse. But yeah I think AR was the intention there
Your phone has an accelerometer, which can be integrated twice to result in slightly accurate dead reckoning; just keep accumulating 3-vectors with every time-slice. That's a helluva lot easier than visual cues.
"slightly accurate" is the correct description, though -- dead reckoning on a cell-phone-class IMU accumulates drift too quickly to keep objects anchored in space for more than seconds at a time. You need something (visual odometry, SLAM) that provides long-term stability, even if it has short-term noise or poor availability, to fuse in.
When this topic comes up, I'm always happy to share this incredible lecture from many many years ago about sensor fusion and dead reckoning and how you _cannot_ integrate acceleration twice:
https://www.youtube.com/watch?v=C7JQ7Rpwn2k&t=1401s (the whole video is incredible, but this timestamp is the magical bit)
2 replies →
In my experience, on Android, it definitely relies on camera: AR doesn't work in dark room, and barely works in slightly-lit room. So it must be fusing camera + accelerometer + ...