Comment by spartanatreyu
7 days ago
We already have "standards" to implement this the web-standards way, but they don't have wide compatibility yet.
1. Use CSS Images Module Level 4's element() function to capture an image of the layer below. (currently only implemented in firefox)
2. Feed that image into an offscreen canvas.
3. Use a shader to distort the image as needed. This can be done in a paint worklet so it doesn't slow down or hold up the main thread.
4. Use CSS Painting API Level 1's paint() function to paint the contents of the canvas onto the background of the button. (currently only implemented in blink based browsers)
How can you use element() to "capture an image of the layer below" and pass it to a canvas?
I might be wrong, but without more context, that sounds like it'd defeat browser protections to avoid leaking your browser history via the color of :visited links.
Oh shit privacy!
I got confused between the <image> css type (that `element()` produces) and the image type used by canvases. But they're different on purpose to try and make things one way to handle privacy/security concerns. (canvases can go in the other way, but they can end up "tainted" and you have to mess with CORS which a whole bunch of devs can't handle)
We can:
- declaratively render <image> css types onto elements from css land using `background`
- declaratively get the finished render of an element back into an <image> css type using `element()`
- programmatically make whatever changes to bitmap data inside a canvas
- programmatically copy the contents of a canvas through Houdini's paint worklets into an <image> css type that is declaratively accessible using `paint()`
We just don't have a non-CORS way to directly get an <image> css type into a canvas' bitmap data in the first place.
I haven't checked, but there could be a way to draw an <image> css type into an svg, then draw that svg in the canvas. Assuming that doesn't break the element() link, you could do the CORS headers dance to make the canvas' data accessible again.
So, you could probably make it work on your own website, but you couldn't release it as a library because each server would need to be set up correctly to allow it to work.
It'd probably be much easier to just skip the canvas entirely and do the distortions entirely declaratively in svg using filter effects (e.g. svgDisplacementMap, or maybe feConvolveMatrix) so there's no privacy leaking.
Ironically, that'd probably mean that the effect could be completely implemented right now in firefox without waiting for new features to be released, but with firefox's poor svg filter effects performance, it'd run at seconds per frame instead of frames per second.