Comment by gfody

the moat is real. the big expensive base models are like the data collected from huge particle accelerators - there's enough unknown structure to be mining for years. you can extract features with more and more generation loss but access to the raw weights is a real advantage, and literally a moat if the interesting behaviors are fenced off