← Back to context

Comment by amluto

3 years ago

I wonder if a compositor, and possibly an entire compositing system designed around adaptive sync could perform substantially better than current compositors.

Currently, there is a whole pile of steps to update a UI. The input system processes an event, some decision is made as to when to rerender the application, then another decision is made as to when to composite the screen, and hopefully this all finishes before a frame is scanned out, but not too far before, because that would add latency. It’s heuristics all the way down.

With adaptive sync, there is still a heuristic decision as to whether to process an input event immediately or to wait to aggregate more events into the same frame. But once that is done, an application can update its state, redraw itself, and trigger an immediate compositor update. The compositor will render as quickly as possible, but it doesn’t need to worry about missing scanout — scanout can begin as soon as the compositor finishes.

(There are surely some constraints on the intervals between frames sent to the display, but this seems quite manageable while still scanning out a frame immediately after compositing it nearly 100% of the time.)

8 comments

amluto

Reply
the8472  3 years ago

For fullscreen apps one can do something even better: skip compositing or buffering entirely. Instead cooperate with the GPU and raster directly into the output buffer ahead of the pixels sent to the display. In wayland that's called direct scanout.

But yeah, for non-fullscreen it helps. See https://github.com/swaywm/sway/pull/5063

mrob  3 years ago

Adaptive sync can only delay drawing, never make it happen sooner. This means it can only harm average latency of response to unpredictable events, such as human interaction. (Individual events may have lower latency purely by luck, because latency depends on the position of the raster scan relative to the part of the screen that needs to be updated, and adaptive sync will perturb this, but this effect is just as likely to make things worse.) The lowest average latency is always achieved by running the monitor at maximum speed all the time and responding to events immediately.

Adaptive sync is beneficial for graphically intensive games where you can't always render fast enough, but IMO this should never be true for a GUI on modern hardware.

  • amluto  3 years ago

    > Adaptive sync can only delay drawing, never make it happen sooner.

    That’s a matter of perspective. If your goal is to crank out frames at exactly 60 Hz (or 120 Hz or whatever), then, sure, you can’t send frames early and you want to avoid being late. But this seems like a somewhat dubiously necessary goal in a continuously rendered game and a completely useless goal in a desktop UI. So instead the goal can be to be slightly late for every single frame, and then if you’re less late than intended, fine.

    Alternatively, one could start compositing at the target time. If it takes 0.5ms, then the frame is 0.5ms late. If it goes over and takes 1ms, then the frame is 1ms late.

  • drewtato  3 years ago

    With text content, most frames are exactly the same. So what adaptive sync can do is delay a refresh until just after the content has been updated. At a minimum, it can delay a refresh when an update is currently being drawn, which would lower the max latency.