Comment by numpad0

The technically correct method to do that would be:

    1. generate/create a mask or a depth map  
    2. apply map to protect regions to keep  
    3. fill unwanted regions with random noise
    4. run the diffusion model 
    5. (optional) run again on the whole image woth fewer steps if the protected regions aren't blending well

The fact that process like this isn't automated suggests that image generation hasn't caught on - though, tbf, I am among ones that still find generated images magically discomforting and I'm negatively biased.