Comment by mrboatshoepants
3 days ago
That first bar chart showing between 15-36 tokens/second is for a 8,192 token context. Thats not nearly enough context to be useful these days
3 days ago
That first bar chart showing between 15-36 tokens/second is for a 8,192 token context. Thats not nearly enough context to be useful these days
To be clear, at 0.75 words per token, that's about ~6,100 words (roughly ~12 pages), which I suppose could be useful, but as other commenters have pointed out, you'd need some kind of orchestrated swarm/hive that could smartly fit payload to within that envelope
Agent swarms have figured out how to mostly work around context window limitations. There's no reason the same ideas can't be applied on a smaller scale, at least in principle.
Could you please paste some links about this that i can further read? Really curious
[flagged]