Comment by yutreer

What you imagined sounds vaguely like dynamicland from Bret Victor.

https://dynamicland.org/

The dots around the paper are encoded programs, and you can use other shapes, objects, or sigils that communicate with the computer vision system.