Comment by echoangle
5 hours ago
Crazy that this approach seems to be the preferred way to do it. How hard would it be to implement the recording in the browser engine? There you could do it perfectly, right?
5 hours ago
Crazy that this approach seems to be the preferred way to do it. How hard would it be to implement the recording in the browser engine? There you could do it perfectly, right?
This is the correct solution. However you'd need someone that knows C++ well, knows Chrome internals, is familiar with video stuff, audio stuff, knows Chromium rendering pipeline, possibly some GPU APIs as well. That person would cost huge amounts of money due to the required knowledge and complexity.
And then you'd need to maintain the code so it works with future Chrome versions.
I did all that. It was hard. I’m not an expert but fought my way though to make all that work. I didn’t get it perfect but I got it pretty good. There’s some weird challenges when you get that deep.
Just as I got it working AI came along which made it pointless then I realised when I got to the bottom that you cannot do it perfectly because it’s not a deterministic renderer. So I called the project to an end.
> AI came along which made it pointless
What? What does AI have to do with anything here?
It can't be that hard...
https://source.chromium.org/chromium/chromium/src/+/main:com...
You can screen share from browser so surely that API?
The purpose seems to be flashy demo videos to sell web-based tools, so rendering unrealistically smooth interactions is sort of the point.
Oh, I thought the purpose was to to build a "copy this saas" app?
You give the agent a URL it records itself going through UX flows, give that video to a coding agent and you have quite a feature.