Comment by sorrow17
1 day ago
Yeah, for a lot of users who control the exported source code, rewriting model to use control flow ops, or simply removing the control flow code is a viable option and solvable. For some other users who want to export the model as-is, the option is either using the (deprecated) torchscript, or just move on and use torch.compile and run your model in Python.
Those control flow ops aren't even supported on many backends. I know tensor rt doesn't support them for example, at least today.
Removing control flow isn't as easy as you'd think for some. It essentially means ripping large sections out of python and into separately implemented c++.