Comment by scottapotamas

It's computer vision focused and might have been suggested previously, but I think Halide is a pretty good/mature demonstration of one way to approach this - writing the algorithm and the execution descriptions as separate passes with access to auto-optimisers and GPU runtimes.