Comment by andrewstuart

7 hours ago

I did this a few years ago. The approach these guys are taking is kinda hacky compared to other better ways - and I've tried most of them.

It works but only in a limited way there's lots of problems and caveats that come up.

I dropped it in the end partly because of all the problems and edge cases, partly because its a solution looking for a problem an AI essentially wipes out any demand for generating video in browsers.

I ended up writing code that modified chromium and grabbed the frames directly from deep in the heartof the rendering system.

It was a big technical challenge and a lot of fun but as I say, fairly pointless.

And there are other solutions that are arguably better - like recording video with OBS / the GPU nvenc engine / with a hardware video capture dongle and there's other ways too that are purely software in Linux that work extremely well.

You can see some of the results I got from my work here:

https://www.youtube.com/watch?v=1Tac2EvogjE

https://www.youtube.com/watch?v=ZwqMdi-oMoo

https://www.youtube.com/watch?v=6GXts_yNl6s

https://www.youtube.com/watch?v=KzFngReJ4ZI

https://www.youtube.com/watch?v=LA6VWZcDANk

In the end if you want to capture browser video - use OBS or ffmpeg with nvenc or something - all the fancy footwork isn’t needed.

7 comments

andrewstuart

Reply
KeplerBoy  5 hours ago

Using OBS won't make your sluggish animation seem buttery smooth though. This seems to be the point of replit's attempt here. Perfect frame pacing.

On top you could use that technique to record at frame-rates higher than native. There's no reason why you shouldn't be able to redraw a basic page with some animations at a few hundred fps.

virtualritz  7 hours ago

> I dropped it in the end partly because of all the problems and edge cases, partly because its a solution looking for a problem an AI essentially wipes out any demand for generating video in browsers.

That is only because your view omits some other problems this solves/products this enables.

There is an incredible ecosystem of tools out the browser land, to create animation.

If you can capture frames from the browser you can render these animations as videos, with motion blur (render 2500 frame for a second of video, blend 100 frames each with a shutter function) to get 25fps with 100 motion blur samples (a number AfterEffects can't do, e.g).

  • andrewstuart  7 hours ago

    There’s a tiny, tiny market for people who would pay for this.

    Also you must understand that chrome is not a deterministic renderer. You cannot get the per frame control because it is fundamentally designed to get frames in front of the user fast.

    They did some work around the concept of virtual time a few years ago with this sort of thing in mind and eventually dropped it.

    • virtualritz  1 hour ago

      > There’s a tiny, tiny market for people who would pay for this.

      Not sure what market you are talking about.

      What I was talking about: people pay for motion graphics. LLMs are excellent at creating motion graphics from/around browser technology ...

      Advertising is a huge market and motion graphics is everywhere in video/film-based advertising.

      > Also you must understand that chrome is not a deterministic renderer. You cannot get the per frame control because it is fundamentally designed to get frames in front of the user fast.

      It absoluetly deterministic if you control the input. There is no "add random number to X" in Chrome. The non-determinism is user inputs and time.

      I know this because the company I work for did extensive tests around this last year. I was one of the people working on that part.

      We looked into the same approach as replit. The only reason we gave up on it was product-related which changed our needs. Not because it is impossible (which, I guess, their blog post prooves).

pjc50  7 hours ago

"Use OBS" is one approach that definitely works. If you run the browser inside OBS it also disables hardware acceleration, which may cause some issues but has the advantage of turning DRM support off.