Comment by jjcm
3 days ago
I'll share my results / my approach. Here are three designs from the prompt->design thing I'm working on:
https://image.non.io/10037610-e35e-44b0-b5c6-69d8fb772109.we...
https://image.non.io/dcf067bc-e296-4744-9b36-2b882f3d791d.we... (same as above, but with your simplified map)
https://image.non.io/94fdfb04-c57d-4b81-8d53-7b0f707e4d63.we...
I've found that starting using diffusion to render your creation, then using a LLM to build from the image creates much less of a slop feel than just starting out with a LLM. You wouldn't tell a construction crew to just build you a house without an architectural plan, so why tell a LLM what visual result you want without a visual guide?
my thing is diffui.ai if you want to check it out. It's basically a harness for diffusion models to generate UI, as well as agent integration for handoff.
That's really nice. Have you tested if it works well with longer and more detailed prompts? For example adding more whole product specs and so on. It would be nice to generate a design system from generated UI you like instead of recreating that UI directly.
RE longer prompts, yes. Generally speaking I expand most prompts to be around a full page of text as it is already, so adding more detail in just refines that expanded prompt. That's more for a single screen though. It sounds like what you're asking about is something like a design.md for an entire brand / docs for a design system.
For that, I have a different approach, which is to extract your design system from screenshots. After which you can just select the brand you want when generating. There's sample images in the sibling comment in this thread.
Also it might be worth noting my pedigree here - I ran the design systems features over at Figma for around 5 years, but quit to build out diffui. The project is heavily oriented towards being able to replicate brands consistently, since the target audience I'm going after is enterprise design teams who are having trouble with existing tools capturing their brand look/feel.
This is an interesting yet simple approach. To the OP’s original question, how might you abstract this into a “design system” that can be applied to their other projects?
I have a "brands" system that's exactly for this: https://image.non.io/27e099c5-f0d7-493a-a01f-928d9e42cef5.we...
Once you have a few reference screens, you can generate a brand guidelines image, which is a visual reference of your brand's look and feel: https://image.non.io/1cc2922a-aec6-4e3c-82c9-895974dd599b.we...
From there you just select the brand at generation time. I've found you don't need a design.md or a npm package - simple screenshots are plenty good enough. Here's a prompt for "a landing page for a new satellite connectivity" feature I generated in reddit/netflix/slack's brands: https://image.non.io/b5e23f19-5041-4f87-9b97-0af39986d1b0.we...