Comment by jerkstate
9 hours ago
I just ran my image recognition benchmark on it ("is this XXX public landmark"?) and it misses a lot that bytedance seed 2.1 turbo gets right; for example: Asked "Is this Salisbury Cathedral" and supplied a picture of Wells Cathedral, it answers "Yes, the west facade of Salisbury Cathedral". Bytedance seed 2.1 turbo correctly says no. Similar results for a picture of Manhattan Bridge sent as Brooklyn Bridge, Chartres Cathedral sent as Notre Dame, etc. I have a benchmark of 12 such images and seed gets 11/12 and deepseek only gets 6/12.
This is a fairly small model for coding and agentic work.
Training it on images like yours would just make it worse in other areas.
> The deepseek-v4-flash-vision-exp model accepts images alongside text, so you can ask the model to describe pictures
it doesn't specify what type of images it can and can't describe, I'm pointing out what type it isn't good at compared to other models.
I have zero interest in world knowledge for my LLMs but this got me wondering : are there RAGs for that kind of data ? How could a LLM like DeepSeek-v4-flash-vision-exp accurately answer you question with an indexed database of labeled landmark pictures (or even 3D models ?).