Comment by mft_
9 hours ago
I've played with diffusion models on and off since the first release of Stable Diffusion - just for amusement, without a particular goal.
Recently, I've been helping a friend's wife with some basic vector images for her sewing hobby (she has what is essentially a CNC sewing machine) and have been super-impressed with FLUX.1-Kontext, which I've been running on my Macbook Pro with mflux. Its ability to (for example) take a photo of a human or an animal and return a line drawing which is recognisably them (rather than just a generic similarish image as I've experienced with other models) is excellent.
It's an older model now, but (AIUI) has the text-handling features baked in, and in my various testing is very reliable at giving me the outputs that I want, without the randomness I've experienced previously. It's big and relatively slow (~3 mins per 512x512 image edit on my M1 Max Mac) but excellent to work with. It's also very straightforward to set up, without the harness complexity of e.g. comfyui.
How are you converting the bitmaps into vector images?
This is currently the weakest part of the workflow. I'm tracing the output images with VTracer [0] and then finishing them manually.
Through a combination of careful initial prompting (e.g. requesting line drawings, requesting limited colours ["posterization" works] if you need colour at all, requesting background removal) and then using the settings in VTracer to limit the colours used for the resulting SVG to the minimum needed, you can end up with an SVG that doesn't take too long to clean up in Inkscape. It usually takes a few minutes of deletions and path unions to produce something usable.
Diffusion models outputting SVGs would obviously be very interesting for this, but this is probably a small niche compared to everything that diffusion models can do with raster images, and I've not found anything usable online so far. LLMs (understandably, given their core technology) are still pretty bad at creating SVG art.
[0] https://github.com/visioncortex/vtracer
is the cnc sewing machine an off the shelf model or something DIY? I'd love to hear more
Off the shelf - it’s a Brother. It prints via a proprietary file format (.PES) but there’s an extension for Inkscape that supports creation and export.