← Back to context

Comment by petu

2 days ago

> and more memory/computationally expensive to run any way I look at it.

Just speculating, but look at attention. 35B-A3B context is heavier -- about 20GB per 1M tokens vs ~7GB. So if you have ≥10M tokens of context per inference node, then DS4 Flash uses less memory than Qwen 35B.

Qwen attention is probably more computationaly expensive as well.

Also I guess not much interest/competition in serving Qwen edge model.