Comment by vbarrielle

There is some long term planning going on, but bad luck when sampling the next token can take the process out of rails, so it's not just an implementation detail.