Comment by tylerhou
1 day ago
It is much easier and more maintainable to convert to continuation passing style. If you also use defunctionalization to allocate closures on a stack instead of a fresh heap allocation for every closure, you will achieve performance on par with an explicit stack. (In fact, defunctionalization is a mechanical transformation the produces exactly the data you would store in an explicit stack!)
Before I knew about CPS and defunctionalization, I wrote a Python decorator that did exactly the transformation you describe. https://github.com/tylerhou/fiber. Now I know about CPS and defunctionalization, I realize that my work was not the best implementation (but it was a fun learning experience!).
It's not just pausing and resuming that I'm looking for, but serialization into a form that can be saved to and loaded from files. (Another commenter calls this "materialization" of continuations.) In that regard, the problem with CPS is that continuations are totally opaque (or nearly so) in every language I'm aware of: you aren't able to do anything with them except resume them within the same execution of the program. Do any of your "defunctionalization" implementations offer such a feature?
Defunctionalization is a general technique that allows one to serialize functions as data. You’ve almost certainly used this technique before without realizing it was an instance of defunctionalization.