← Back to context

Comment by njoyablpnting

15 hours ago

> On Remotion, yeah, not sure it's the right fit, but honestly the sheer capability of models at writing code these days has surprised me.

Just to clarify I still think code-driven graphics is the correct approach, but in my case I opted for a different library with a more powerful imperative API.

> Also noticed they have an experimental client-side rendering version built on mediabunny

Yes, I've tried it out, it was a non-starter for me because it only supports canvas-based components, and Remotion didn't seem to have good support for text on canvas because they rely on HTML for most of that.

> On WebCodecs, there are a fair set of challenges, but we wanted to take the bet

Totally understand the appeal and immediacy of a browser app, I was lured in by that too. For what it's worth I've reported showstopping WebCodecs issues in Chromium and there's basically no indication they'll get fixed on a predictable timeline.

Another issue I ran into that I just remembered is animating text on canvas. It's basically impossible to get pixel-perfect anti-aliased text animation using a canvas. I would have to dig up the exact details but it was something to do with how browsers handle sub-pixel positioning for canvas text, so there was always some jitter when animating. This coupled with the aforementioned WebCodecs issues led me to conclude that professional-quality video rendering is not currently possible in the browser environment. Aliasing, jitter and artifacts are immediately perceptible and are the type of thing that users have zero tolerance for (speaking from experience).

This is not meant to be discouraging in any way, I've just been very deep into this rabbithole and there are some very nasty well-hidden pitfalls.

5 comments

njoyablpnting

Reply
spuzvabob  12 hours ago

> Totally understand the appeal and immediacy of a browser app, I was lured in by that too. For what it's worth I've reported showstopping WebCodecs issues in Chromium and there's basically no indication they'll get fixed on a predictable timeline.

Interestingly I have the exact opposite experience, I've reported issues both in the WebCodecs specification and the Chromium implementation, in all cases they were fixed within weeks. Simply though reports on public bug trackers and it wasn't really a major issue in any instance.

> Another issue I ran into that I just remembered is animating text on canvas. It's basically impossible to get pixel-perfect anti-aliased text animation using a canvas. I would have to dig up the exact details but it was something to do with how browsers handle sub-pixel positioning for canvas text, so there was always some jitter when animating. This coupled with the aforementioned WebCodecs issues led me to conclude that professional-quality video rendering is not currently possible in the browser environment. Aliasing, jitter and artifacts are immediately perceptible and are the type of thing that users have zero tolerance for (speaking from experience).

We're doing SOTA quality video rendering with WebCodecs + Chromium with millions of videos produced daily, or near SOTA if you consider subpixel AA a requirement for text. In general for pixel perfection of text, especially across different browsers and operating systems, you can't just use text elements in DOM or in canvas context, instead text needs to be rasterized to vector shapes and rendered as such. Honestly not sure about potential jittering when animating text, but we've never had any complaints about anything regarding text animations and users are very often comparing our video exports with videos produced in Adobe AE or similar.

  • njoyablpnting  9 hours ago

    > Interestingly I have the exact opposite experience, I've reported issues both in the WebCodecs specification and the Chromium implementation, in all cases they were fixed within weeks. Simply though reports on public bug trackers and it wasn't really a major issue in any instance.

    That's fair, they are responsive most of the time. I do have one major rendering issue in particular I've been waiting on with no movement for months, so I might be biased.

    > We're doing SOTA quality video rendering with WebCodecs + Chromium with millions of videos produced daily, or near SOTA if you consider subpixel AA a requirement for text. In general for pixel perfection of text, especially across different browsers and operating systems, you can't just use text elements in DOM or in canvas context, instead text needs to be rasterized to vector shapes and rendered as such. Honestly not sure about potential jittering when animating text, but we've never had any complaints about anything regarding text animations and users are very often comparing our video exports with videos produced in Adobe AE or similar.

    So you use a library that takes in text and vectorizes it to canvas shapes? That could work in theory, do you have a demo of this?

    • spuzvabob  9 hours ago

      > So you use a library that takes in text and vectorizes it to canvas shapes? That could work in theory, do you have a demo of this?

      Yea, it's harfbuzz compiled to WASM: https://harfbuzz.github.io/harfbuzzjs/ Then all text layout features must be implemented on top of it, like linebreaking, text align, line spacing, kerning, text direction, decoration etc.