Comment by skerit

5 days ago

I see Ghostty does not support (and does not plan on adding support for) Sixels, instead preferring the Kitty image protocol.

Now if the Kitty image protocol is so great and the Sixel stuff is so bad, ~~why is it only used in Kitty and Ghostty?~~

*Edit: it's also supported in Konsole, WezTerm, ... but still I'm interested in why we have 2 competing protocols right now.

41 comments

skerit

Reply
shiomiru  5 days ago

Sixel came earlier, and already fulfilled the basic requirement of "put pixels on screen in a single well-defined format" (something not even iTerm2's protocol does.)

Kitty is a lot more complex: it accepts five different encodings, has three different ways to load the data, supports animations, etc. So it's no wonder only a few terminal developers had time to implement it.

See also: https://github.com/veltza/st-sx/issues/1#issuecomment-190272... 5000 lines (Kitty) vs 1000 lines (Sixel) even though the Kitty patch is just a "subset".

embedding-shape  5 days ago

> Now if the Kitty image protocol is so great and the Sixel stuff is so bad, why is it only used in Kitty and Ghostty?

Images as in "pictures" or is that something else? I'm using Alacritty, and I don't think I've once thought "I need to see this image inside the terminal" and I do deal with images and frames from videos a lot. Probably if I saw it being added to Alacritty I'd think it was adding unnecessary bloat, so I wouldn't be surprised not every terminal is rushing to implement it.

Or I completely misunderstand what you're talking about.

  • ziotom78  5 days ago

    I run Kitty and use this feature regularly. Most of the time, I rely on it within Yazi [1], a TUI file manager, but I can also display plots within the Julia REPL, thanks to the KittyTerminalImages.jl package [2]. It's even more crucial when I'm navigating a remote directory and need to check an image file, as I usually have timg [3] installed on those servers. Once you discover how valuable this is, it becomes a permanent part of your workflow.

    [1] https://yazi-rs.github.io/

    [2] https://github.com/simonschoelly/KittyTerminalImages.jl

    [3] https://github.com/hzeller/timg

  • xp84  5 days ago

    I have to say, (caveat, I have not tried any of these yet) that I am intrigued by all these features like graphics being added to terminals. It feel like exploring an alternate timeline 1990s where the GUI “lost” — of course in the absence of a successful Windows and Macintosh, terminals would have naturally gained these graphical abilities 30 years ago.

  • setopt  5 days ago

    It’s pretty nice to ssh into a remote host and plot some data there without needing either X forwarding, or dumping to files and rsync’ing, or similar workarounds.

  • porridgeraisin  5 days ago

    Yes, pictures. It's quite useful. Opening images on remotes for one. Viewing plots arbitrary python scripts create for another.

    Off the top of my head.

  • the_gipsy  5 days ago

    The alacritty maintainers reject any image protocols as unnecessary. I am fond of images in terminals, but I gotta say that I respect their decision, very good call. Not every terminal emulator should do the same.

    • embedding-shape  5 days ago

      I didn't know, but I'm happy to hear we seemingly are aligned regardless :) Thanks for the additional context!

  • trenchpilgrim  5 days ago

    Viewing an image in a terminal can be really handy for debugging ML systems that use images or bitmaps. You can also paste images directly into claude code as context.

    Once while working on a daemon that did both ML and DSP on live audio I added the ability to play sounds and display spectrographs of in-memory audio data at various points of the internal pipeline to debug an issue that would have been difficult otherwise. Way quicker than dumping WAV files to view externally.

  • lloeki  5 days ago

    I'm not particularly fond of displaying images on the terminal in ways that would resemble a GUI.

    That said, augmenting a shell-based workflow with tidbits such as this had me sold:

    https://xcancel.com/thingskatedid/status/1316074032379248640...

    https://xcancel.com/thingskatedid/status/1316075850580652032...

    To achieve that, either Sixel or Kitty protocol is fine. IIRC Sixel works over SSH without any fuss, dunno about Kitty.

    • cb321  5 days ago

      I can confirm that I just do it over ssh fine all the time -- gnuplot, img2sixel as an "image-cat", etc. (`st` with patches to add sixel support as discussed in various places in these comments.)

  • alwillis  5 days ago

    I’ve found it useful, when paired with a terminal file manager, to preview graphics in the terminal.

  • bee_rider  5 days ago

    It would be nice if matplotlib or Octave could display pretty plots and figures on a remote server, in the terminal.

didibus  5 days ago

I think it's because Kitty and Ghostty are the newest terminals, so they came up with new modern options and solutions.

mechanicum  5 days ago

More than two, e.g. there’s also the Inline Images Protocol supported by iTerm2 and WezTerm.

Kovid documented his rationale at some length here: https://github.com/kovidgoyal/kitty/issues/33

IshKebab  5 days ago

IMO none of them are particularly useful. Sixels is hilariously inefficient. Kitty is slightly better because you can send data as PNG, but ... you have to send image data as PNG!

I wish there was a high performance way of remoting graphics over SSH. How cool would it be if you could SSH to a remote machine and it just showed you the remote desktop in the terminal itself? No messing around with port forwarding, weird X servers, etc.

I think probably that requires a full fat video codec like H.264 to work well though. Or maybe RDP?

Probably too many GUI naysayers and "What's wrong with remote X?" for this to ever happen though.

  • zokier  5 days ago

    One of my pet proof of concept projects is figuring out how to ergonomically tunnel web apps over ssh without needing to fiddle with listen ports and port forwards. First attempt was to push http2 over stdio which actually worked, but it didn't really integrate well with terminal use. Currently I think similar approach to X forwarding makes sense, where SSH forwards one unix socket over ssh connection and then the applications can connect to that socket and put http2 traffic over that connection. Basically the idea is to make webapp tunneling as easy as X tunneling, so you can just type command in shell and (browser) window would pop open without any extra hassle. The neat thing is that because http2 has persistent connections with multiplexing etc built in, it works really well for this sort of hack; plain http 1.0 would be far more annoying.

  • toast0  5 days ago

    > I wish there was a high performance way of remoting graphics over SSH. How cool would it be if you could SSH to a remote machine and it just showed you the remote desktop in the terminal itself? No messing around with port forwarding, weird X servers, etc

    I think there's at least three different experiences here, and they're all valid, but I don't know what you really want.

    A) remote desktop --- connect to a fully formed desktop environment (with SSH to authenticate, I guess?), possibly persisted and/or shared so you can connect back and get into the same place or share with another user?

    B) run a program remotely and display it on your local terminal; essentially remote X, but I gather you're looking for more performance and maybe some other nice to haves? Maybe you want to transport audio too... Maybe you don't want the crap experience remote X has become since app developers don't spend any effort on it and you kind of get what you get, which is a lot of jank.

    C) images in the terminal, with high performance. PNG should be ok for that, right? Maybe an extension for lossy compression might be nice depending.

    • IshKebab  4 days ago

      Yeah I want all of those. Only the third one really works at the moment but it's also the least useful. The second one I think would be the most transformative because then all the TUI apps people use like htop, ncdu and so on can have decent graphics.

  • duskwuff  5 days ago

    At that point you're really better off using some other remoting protocol instead of trying to tunnel it all over a terminal session. There's nothing left of the original terminal.

    • IshKebab  5 days ago

      There is though - the ssh authentication and connection is already handled, and I'm already in a terminal. When I quit the app or session I'm back in the terminal.

      If it worked it would greatly reduce the hassle.

      Think about all the TUI apps that exist. They're useful because they're convenient when working in a terminal, not because they look like shit.

      8 replies →

  • anthk  5 days ago

    drawterm under Unix clients and 9front cpu connections; but that's Unix Philosphy 2.0.

forgotpwd16  4 days ago

>why is it only used

Sixel has existed for 40y, Kitty protocol originated and initially made for a single project (Kitty) few years ago.

>why we have 2 competing protocols

Well, iTerm2 also got its own image protocol (predating Kitty's but basic, only meant to show images rather graphics in general) that has been adopted by few other projects. Terminology also got something on its own, and urxvt can show images by setting the background image.