Comment by vunderba
8 hours ago
So thoughts
Positives
• It's a heck of a lot smaller than Qwen-Image 1 (20b parameters) at only 7b, making it one of the smaller open-weight models available (Z-Image Turbo is one of the few that is smaller at 6b) when compared to Ideogram, Krea2, Flux2, etc.
• It supports native transparency (Qwen's team, as far as I know, is the only one attempting to tackle this). Even though it's relatively trivial to set up background removal postprocessors, it's also neat to see it natively supported.
• It's fast using QwenImage2.1 convrot, a 1MP image took around ~5 seconds on an RTX4090.
Negatives
• The license (assuming you respect it) is far more restrictive. The original Qwen Image 1 was released under the standard Apache license; this one explicitly forbids commercial usage without obtaining a separate license. On the other hand, a lot of us didn't expect the Qwen team to ever release "weights-available" ever again.
Qwen-Image 1.0, released about a year ago, only scored 4/15 on my GenAI Showdown Benchmarks. Since that time, they've been upstaged by Krea 2 (6/15) and Ideogram4 (8/15). I'll post the new results once I have some more time to run them.
Well, the results are in, at least for text-to-image (the editing bench will come later).
Qwen-Image 2.1 is definitely a pretty big leap over the last open-weight version, Qwen-Image 1.0, released back in August of last year and managed to score 7 out of 15 as opposed to its predecessor which scored 4 out of 15.
Even though it's significantly smaller, 7b vs 20b, it's multimodal (so you don't need a separate image-to-image model like you did with Qwen-Edit), more coherent, and significantly faster even when outputting at higher 2K resolutions. However, in my testing, I found that I had to play with dialing up the CFG depending on the complexity of the prompt.
I've also added a progress dropdown under Model Performance so you can see how cloud vs. local models have been trending since 2024. Spoiler: June of this year released some of the biggest bangers (Krea 2, Ideogram 4, and the kind of slept-on Boogu-Image 0.1).
Downsides:
- It was clearly trained on at least some level of synthetic training data, and it shows in some of the subpar outputs in terms of fidelity. Some of this you might be able to iron out with a refiner model downstream or a custom LoRA but time will tell.
- They've moved away from the permissive Apache license. Commercial usage is only allowed by request.
Comparisons:
https://genai-showdown.specr.net
If you just want to compare local models only:
http://genai-showdown.specr.net/?models=local
> and the kind of slept-on Boogu-Image 0.1
Not slept on at all. It was absolute trash, and I’m super curious why people pretend otherwise. There isn’t a single thing that model did better than any temporal peer.
That benchmark might have some issues. You prompted the models to generate an image of striking a ring against a crucible. Then you (presumably, manually?) scored the images that depicted an anvil higher than the ones striking something resembling a crucible.
That’s a good catch. Yes, all scoring is done through manual review since relying on a VL model for these kinds of meta-metrics is a sort of loose equivalent of gödel's second incompleteness theorem.
I’ll have to think about this one. When I crafted the prompt, I wasn’t really thinking about the differences between a crucible and an anvil. It was more the visual of an archangel smelting halos for newly arrived heavenly beings.
I'm not sure why one would even strike metal against a crucible! It's a container for liquid metal. One of the outputs shows it being smashed by the manoeuvre, which is probably the most realistic outcome of all of them.
Sorry, I'm not trying to nitpick. I'm just joining in because I'm interested in how the models dealt with the request.
1 reply →
It would make more sense to compare with Qwen Image 2, since that was the last open weights Qwen model.
Edit: This is wrong.
Wait... is that true? I don't think the original Queen Image 2.0 was ever released beyond an API. At least, I don't remember a public weights release.
I promise I searched on my phone before posting that, but it looks like I misinterpreted an hf search result that was actually Qwen-Image-2512 but the url was truncated to Qwen-Image-2 due to my screen width (ugh). For some reason I was thinking Qwen Image 2.1 was the one that had been available for a while but not released, but you're right that was 2.0.
Native transparency isn’t so hard to do by the way, I made an image AE (I don’t say VAE deliberately as none of these are VAEs, I don’t know why they keep being called that since the variational part is completely absent) that supported this about two years ago as a hobby project. I haven’t really been following the space recently, I’m surprised it’s taken so long for this to come out if it’s a first.
Could you add new OAI 2.5 image models?
Can do! GPT-Image-2 already scored unsurprisingly very high: 12 out of 15 on text-to-image, and 10 out of 12 on image-to-image.
The three benchmarks it failed on (D20, Flat Earth, and Banded Snake) are pretty difficult, so I'd be surprised if 2.5 manages to pass them, but I’ll add it for completeness’ sake later this week.
They're trying to cash in but this is just sad
Within a few days this seems a total pivot from Xiaomi’s op RL dashboard and the praise of Chinese open model? What is the sentiment now?
Everything is combined and uneven, including the opinions of hackernews commenters?
There is no single opinion, and clearly no single Chinese approach.
Also Chinese labs are in particular very careful about anything which be used to create pornographic content, which is highly illegal in the PRC.
2 replies →
Qwen can't train anymore with openai reasoning tokens? I kid I kid.