Comment by gfody
1 day ago
the capability gap is underestimated. the raw unaligned base model from a pretrain is like the telemetry recorded from a particle accelerator - it's hugely valuable and not just because of the cost sunk building a collider.
the things are kept under air-gapped national weapons grade security measures not because it's literally going to escape and threaten the world but because if somebody walked out the door with a copy they would have everything. the capabilities we see at the surface are mostly the result of mining the great unknown space and attaching feeble control surfaces, ablating/lobotomizing dangerous areas, and fencing off illegal/secret/embarrassing ones.
> the capability gap is underestimated.
Based off what? There’s pretty precise measurements of the capability gap where open weight models like Kimi K3 score higher than the latest flagship models just 6 months ago.
> the capability gap is underestimated.
The capability gap is in the minds of the users but the frontiers' closed business models run opposite to it.
> the things are kept under air-gapped national weapons grade security measures
That's an enhancement of closed but not of moat
> because if somebody walked out the door with a copy they would have everything.
Including you and me? I'm not sure this is comforting news, despite the "trust me bro" asurances from behind the door where we can't see or verify anything.
> Including you and me?
I don't know about you but I would love to get my hands on a raw frontier base model - and a 16 node galaxy blackhole supercluster to talk to it. the capabilities currently being loboptimized for are just a narrow market-shaped slice that cheaper models can distill a competitive subset of but the untapped power of the base is a true technical moat.
about the benchmarks: the difference between a distilled competitive subset and the untapped base might be the difference that matters in any given task
FWIW, I meant "they would have everything... including you and me". It seems I wasn't clear enough and it sounded like "you and me... walk out the door with a copy" - obviously the latter isn't realistic, but the former is.
> the capabilities currently being loboptimized for are just a narrow market-shaped slice
Offensive capabilities aren't interesting to me except as a risk I have to consider. It might sound surprising but the intersection of offensive and practical-for-life capabilities is an almost empty set.