Comment by theteapot
5 hours ago
Dumb question: When your testing "ChatGPT 5.6 Sol" are you testing an actual LLM or some visual pre-processor stack that sits in front of it (along with a maybe a bunch of other such pre-processors) that is bundled into what's call "ChatGPT 5.6 Sol"? I.e. last I checked LLMs had a something like a 30-100K token alphabet to work with and it's hard to imagine how throwing pixels arrays at one directly would work.
I'm ~95% certain that images are tokenized, just like regular text, and fed directly in; that's the 'multimodal' part of these models. Now how this tokenizing works I don't know, and there might be some level of preprocessing, but it's certainly not converting the image into text and feeding it in to a regular LLM.