Comment by cmrdporcupine

Ok I tried it. And it's faster. Though I'm still in verifying quality phase.

But only works for Gemma4, or other attention-only models. (i.e. not the Qwen3.8 models I had working with the other way)

I'm getting 63ms per answer for 1 question, 79 for 3, 93.7ms for 5. So scales nicely, too. ~55ms fixed cost + ~7-8ms per additional question.

Much better. Thanks. I'll commit and share work after testing it some more.

EDIT: how can I credit you in the commit body?