Comment by mark_l_watson

Have you used Nemotron-3.5-lightening? I don’t use it as much as Poolside’s (excellent!!) Laguna XS 2.1 6bit, but the new Nemotron model is good.

I think NVIDIA does want small open models running on-prem to explode as a market! Lots of smaller GPU installations for companies who wisely want on-prem inference.

Of course NVIDIA will also keep making a ton of money selling to hyper scalers, but not forever: Chinese chips are getting better, Google, Microsoft, Amazon, etc. designing their own inference chips.

NVIDIA is handling this brilliantly.