Comment by sothatsit

The bigger change is just to manage multiple contexts at all. I think how that is implemented will be determined through experimentation. I don't think the problems get much harder when you have multiple API requests in flight at once vs. doing them serially as you suggest. And for today's models, the speed increase would be nice, so it seems like it would be worthwhile.