← Back to context

Comment by dmos62

2 days ago

I've seen a graphical designer struggle to use an analogous product. They tried prompts like "remove background from this photo of dancers, don't change their face": the model couldn't do it: it always changed the faces at least a bit. In other words, the model was not removing the background, instead it was generating the whole picture anew, but without the background, which not only seemed inefficient, but was also way too lossy.

I wonder if this model could finish that task.

7 comments

dmos62

Reply
numpad0  2 days ago

The technically correct method to do that would be:

    1. generate/create a mask or a depth map  
    2. apply map to protect regions to keep  
    3. fill unwanted regions with random noise
    4. run the diffusion model 
    5. (optional) run again on the whole image woth fewer steps if the protected regions aren't blending well

The fact that process like this isn't automated suggests that image generation hasn't caught on - though, tbf, I am among ones that still find generated images magically discomforting and I'm negatively biased.

dvrp  2 days ago

Did you check the “Only what you asked. Not one pixel more” section?

Not only do I think the website section addresses exactly this, but we also built a pixel diff viz tool for you to compare with other models.

  • yorwba  2 days ago

    That section shows a single example, where pixels in some rectangular region around the selected object are being changed. For background removal, that rectangular region would encompass the entire image, so it's not clear whether you have any particular advantage there.

coolius  2 days ago

I wonder if your friend would have been more successful by removing "don't change their face" from the prompt

  • hdjrudni  2 days ago

    A lot of models required exactly that. But I imagine it varies from model to model.