← Back to context

Comment by johndough

11 hours ago

Might still be fine. The most recent crop of vLLMs proactively use whichever programs are available on the system (e.g. ImageMagick or PIL) to "zoom in" by cropping subimages if they can't quite make out the details.

13 comments

johndough

Reply
knollimar  8 hours ago

Downsizing a higher res image to lower res means the zoom will be blurry.

  • andai  8 hours ago

    They process the original image file with Python on the local device. (And I've seen the web chats do this with their "computer use" features too.)

    The really wild one is even blind models will do this and they'll try to run stats on the pixels to figure out what it looks like... the even wilder thing is that it kind of works!

    • knollimar  7 hours ago

      If the API accepts only 800 by 800, the aegument youre making is "fix it in the harness".

      I don't think the n by n subgrid fixes this the way most harnesses do, as it'll fail to count things if you have more overlap and fail relatiomships if you have less

      1 reply →

  • johndough  5 hours ago

    The order is:

        LLM issues tool call to read high res image ->
    harness sends high res image to server ->
    server downsizes it to 800x800 (blurry) ->
    LLM issues bash command (e.g. `convert`) to crop a small subimage (e.g. 600x600) from the high res image ->
    LLM issues tool call to read subimage ->
    harness sends subimage to server ->
    server does not resize the subimage because it is small already, so it is not blurry when finally ingested by the LLM

  • adastra22  6 hours ago

    They’re not talking about zooming, hence the quotes.

    • johndough  5 hours ago

      Yes. When the LLM tries to read an image, it will be resized by DeepSeek's server to 800x800, which might be a bit blurry. The LLM will then crop a smaller image from the high resolution image (using e.g. the `convert` tool via bash) and will then read the small cropped image. This image will still be resized to 800x800 by DeepSeek's server, but since it is already small, there is no or little loss of quality.

    • knollimar  6 hours ago

      If the harness does it that's just like saying "please use a workaround". You'll lose fidelity and LLMs will lose the ability to count things or maintain relationships for schematics, etc

      3 replies →