Comment by qwertox
2 years ago
This is my main issue, I need full access to input devices and the ability to drop, alter and inject new events.
On Windows I fully rely on StrokeIt, it has been a huge improvement for me which I've used for ~15 years.
Linux's easystroke [0] is now deprecated (and always was mediocre) and the modern alternative "mouse-actions" [1] is not good at all.
I've coded a replacement in Rust+C which works for my simple use-case but it targets X11 and it's unclear for me how I can get low level access to input devices events with Wayland. I've also read something about security getting increased in Wayland so that one does no longer have access to input stuff (unclear to me).
Then the nice thing about X11 is the ability to remotely use a X11 session, which apparently isn't possible with Wayland (except for RDP or VNC which isn't what I need).
> ... and it's unclear for me how I can get low level access to input devices events with Wayland.
You can't as far as I know, the reason it has taken more than a decade to gain adoption is the protocol design turned out to be unusable for desktops (at least, probably other platforms too). I've been avoiding it because Zoom & friends didn't work when I tested them. My understanding of the de-facto situation is that everything sucked for about a decade from the release, then (special mention to Drew DeVault) people started extending the Wayland protocol to cover the original design mistakes.
Documentation is available [0]. If I were trying to understand the state of play for mouse gestures I'd probably start by researching what is going on the wlroots project, that seems to be where a lot of the useful innovation was coming from. The extensions implemented in the compositor look like they matter a lot.
[0] https://wayland.app/protocols/
Zoom and friends didn't work, because they thought they can grab root window content and run with it. By a chance it did work in X11 due to how PC hardware worked and thus Xorg organized it's framebuffer, but doesn't neither Wayland (for obvious reason), nor when hardware uses it's newer features, like overlays. Even on other platforms, they have to ask, and the user might permit that. They just didn't bother to ask on Linux (it is done via pipewire), and didn't bother to fix their approach.
But that gets back to the basic issue here - the answer to "how should Zoom take screenshots if someone is using a Wayland compositor?" turns out to be "Well, you needed to implement a separate server that re-imagines video as streams, like PulseAudio. Then they have to interface with that. Of course, Wayland doesn't support that (it explicitly makes it hard, in fact), so someone needs to design a protocol extension [0] first to make all this work".
That is a very complex answer. On X, the answer is "grab the relevant window". I can see why Zoom gave the whole mess a pass until the community sort themselves out. I'm doing the same thing; although it looks like (checks) 15 years of patience has been rewarded and Wayland isn't too much of a backslide from X servers.
Unfortunately automating I/O still isn't all that clear, I'd have clicking automated before I move over; it probably is possible but it is just a messy situation to figure out how.
[0] Technically not a Wayland extension, they bypassed the entire system. So now "Wayland compositor" tells us nothing whatsoever about whether the system can take screenshots, but we have to look up compositor-specific support for screencapture. The situation is ... technically better than X, but still silly. They could have avoided years of wasted time by just acknowledging that people want to intercept data as part of the protocol. The situation is a mess and it is silly that we can't have generic Wayland screenshot applications.
1 reply →
> I've been avoiding it because Zoom & friends didn't work when I tested them.
If you are talking about screen sharing Zoom, Meet and Slack work fine on Debian stable running Wayland in KDE and Gnome. Teams probably doesn't work on Wayland, but then Teams working on any other platform on a given day is at best a lottery.
There's no native Teams for Linux anymore, only the browser version. Browsers have no issue working with Wayland/pipewire.