Yeah reading this I was thinking, we've got Qwen-Image-Edit which is an image model with an LLM backbone that takes well to finetuning.
I'd be surprised if you can't get a 80%/20% result in a weekend, and even that probably saves you some time if you're just willing to pick best-of-n results
The person behind www.pixellab.ai has been trying to make a SaaS out of that idea for about 2 years already and it just isn't there, the examples in the homepage are extremely cherry-picked, I bet most of their paying customers just use it as a starting point and then spend hours manually fix the sprites, which may be more than enough value for $12 a month and that's great but what is shows is that we are not as close as one would like to imagine, the "one leg in front of the other at about the same depth and the same color" is still problematic to this day; if most pants in the world had a different color for each leg I bet most of its animation issues would be solved, unfortunately we don't and most of the training data involves single-color pants/legs.
At the risk of sounding unfairly stubborn about something I'm not that familiar with, if they've been at it for 2 years I'm imagining a very different (much more difficult) pipeline than fine-tuning an image model with an LLM backbone
Yeah reading this I was thinking, we've got Qwen-Image-Edit which is an image model with an LLM backbone that takes well to finetuning.
I'd be surprised if you can't get a 80%/20% result in a weekend, and even that probably saves you some time if you're just willing to pick best-of-n results
The person behind www.pixellab.ai has been trying to make a SaaS out of that idea for about 2 years already and it just isn't there, the examples in the homepage are extremely cherry-picked, I bet most of their paying customers just use it as a starting point and then spend hours manually fix the sprites, which may be more than enough value for $12 a month and that's great but what is shows is that we are not as close as one would like to imagine, the "one leg in front of the other at about the same depth and the same color" is still problematic to this day; if most pants in the world had a different color for each leg I bet most of its animation issues would be solved, unfortunately we don't and most of the training data involves single-color pants/legs.
At the risk of sounding unfairly stubborn about something I'm not that familiar with, if they've been at it for 2 years I'm imagining a very different (much more difficult) pipeline than fine-tuning an image model with an LLM backbone
The jump in understanding that having a full sized LLM behind the generations enables here is massive: https://ghost.oxen.ai/fine-tuned-qwen-image-edit-vs-nano-ban...