Comment by Double_Cast

Ideally, the mapping from object to image is injective. What you're proposing will lead to "hash collisions" aka blurriness, since each point in the object will bleed colors into neighboring points in the image.

The entire object doesn't get collapsed to a single point. Rather, a single point of the object radiates light in all directions. A lens then captures a fraction of that radiation and collapses it back down to a single point. Then we iterate over each point in the scene with a "for each".