Comment by search_facility
5 hours ago
They are used a lot, and modern edit models kind of support base cases - depth, canny, openpose - as a reference image. I mean they understand "use image 2 as depth map", result indeed have same depth, etc. But interpretation sometimes arbitrary to model.
For the tasks where several control nets should be adhered (depth and canny and segmentation) dreaded SDXL is still unmatched. SDXL also the only one handling influence weight for each control net normally, this ability completely vanished on modern models
Control nets in early post-SDXL models like ZIT or Flux kind of work too, but still much worse in quality when combined. I suspect pure diffusion architecture of SDXL is really favorable to mixing control signals with precision, DITs probably can be trained to similar level for mixing signals - but it is much harder and never seen in practice. At least every post-SDXL model I tried is trained with single one in mind, judging from my attempts. This rant is from personal expirience.
No comments yet
Contribute on Hacker News ↗