← Back to context

Comment by catlifeonmars

17 days ago

I’m really surprised that GC is an issue at the bits/sec throughput a TUI would be pushing. At the risk of making an obvious observation: your render loop is doing way too much work for what it is producing.

15 comments

catlifeonmars

Reply
chrislloyd  16 days ago

Most people's mental model of CC is that "it's just a TUI" but it should really be closer to "a small game engine". For each frame our pipeline constructs a scene graph with React -> layout elements -> rasterize them to a 2d screen -> diff that against the previous screen -> _finally_ use the diff to generate ANSI sequences to draw. We have a ~16ms frame budget so we have roughly ~5ms to go from the React scene graph to ANSI written. You're right that in theory we shouldn't have to do much work, but in practice that's required optimizations at every step.

For the GC pauses specifically, what mattered is predictable performance. More allocations == more GC == more frames where the VM is locked up seemingly doing nothing. On slower machines we were seeing this be in the order of seconds, not ms and when somebody is typing all they feel is the 1 character that's stuttering. Honestly, I was surprised about this too as GC in JS is often not something that's too impactful.

  • vintagedave  16 days ago

    Can I ask why you used JavaScript at all for CC? Or even React for a simple UI? It seems misaligned with the app’s nature. This bug, GC pauses, everything else you mention… This isn’t criticism, because I believe people make good judgements and you and Anthropic will have good reasons! It’s just puzzlement, in that I don’t understand what the balance judgement was that led you here, and having experienced all the issues it led to I would love to know what the reasons were. Thankyou for any insights you can share :)

  • catlifeonmars  16 days ago

    Thanks for the in depth explanation! I think the comparison to a game engine makes a lot of sense. Is the diff just part of the react rendering engine, or is it something you intentionally introduce as a performance optimization? Mostly I’m wondering how much the diff saves on rendering performance if you’ve already generated a 2D raster. In the browser, this saves on API calls to the DOM but at that point you haven’t rendered anything resembling an image. Is this what helps to resolve flickering, perhaps?

  • withinboredom  15 days ago

    > On slower machines we were seeing this be in the order of seconds, not ms and when somebody is typing all they feel is the 1 character that's stuttering.

    You mean like a laptop that is trying to stay cool (aka, cpu throttling) on battery power while Claude is running a slightly different version of the test suite for the 5th time to verify it didn't break anything?

    Yeah, the typing latency is really bad in those cases. Sometimes waiting for 40 seconds or more.

moltar  17 days ago

I think this is the main issue. When I would get into flickering mode, it appeared that the entire TUI was re-rendering on every key press. I don’t know if it’s maybe just the limitation of Ink or even terminals.