Comment by tptacek

1 day ago

This is a tangential point (this post is not really about TUIs; sort of the opposite) and I think lots of people know it already but I only figured it out last week and so can't resist sharing it: agents are good at driving tmux, and with tmux as a "browser", can verify TUI layouts.

So you can draw layouts like this and prompt Claude or Gemini with them, and get back working versions, which to me is space alien technology.

19 comments

tptacek

Reply
Kerrick  11 hours ago

I've had a lot of good luck with AI Agents also being able to read and meaningfully interprete .txt and .ansi dumps from RatatuiRuby's TestHelper module and its `assert_snapshots` [0].

For example, it was able to read the ANSI and figure out that a snapshot had changed because an upstream bug properly rendered bold and underline when it hadn't before.

[0]: https://git.sr.ht/~kerrick/ratatui_ruby/tree/trunk/item/lib/...

agavra  1 day ago

This is spot on, I understand very little about how terminal rendering works and was able to build github.com/agavra/tuicr (Terminal UI for Code Review) in an evening. The initial TUI design was done via Claude.

eterps  1 day ago

Would love to hear more about this approach.

  • tptacek  1 day ago

    It's actually really easy in Claude Code. Get a TUI to the point where it renders something, and get Claude to the point where it knows what you want to render (draw it in ASCII like this post proposes, for instance).

    Then just prompt Claude to "use tmux to interact with and test the TUI rendering", prompt it through anything it gets hung up on (for instance, you might remind Claude that it can create a tmux pane with fixed size, or that tmux has a capture-pane feature to dump the contents of a view). Claude already knows a bunch about tmux.

    Once it gets anything useful done, ask it to "write a subagent definition for a TUI tester that uses tmux to exercise a TUI and test its rendering, layout, and interaction behavior".

    Save that subagent definition, and now Claude can do closed-loop visual and interactive testing of its own TUI development.

    • electroly  1 day ago

      Can you explain tmux's contribution here? I'm confused why this process wouldn't work just the same if CC directly executed the program rather than involving tmux. Are you just using tmux to trick the program under test into running its TUI instead of operating in a dumb-stdout mode?

      11 replies →

frumplestlatz  17 hours ago

I’ve actually got an MCP server that makes it really easy for Claude to generate key events, wait for changes / wait for stable output / etc, and then take PNG screenshots of the terminal state (including all colors/styling) — which it “views” directly as part of the MCP tool response.

Wish I could open source it; it’s a game changer for TUI development.