Comment by kristianp
14 days ago
I'd like to discuss the matter of size. Llama has gone from talking up an 8b model as capable to having a smallest model of 109b. What will be the sizes in a years time? Things are moving out of reach for commodity pc's, 128GB is possible, but expensive.
I'm hoping that Llama 4 goes the same way as Llama 3.
The first Llama 3 models released were 8B and 70B in April 2024.
Llama 3.1 came later in July at 8B, 70B, and 405B.
Llama 3.2 in September got really interesting: 1B, 3B, 11B and 90B.
Then Llama 3.3 in December was 70B but claimed performance similar to the earlier Llama 3.1 405B!
Llama 4 is 109B and 400B, both of which were trained with the help of the 2T(?) "Behemoth".
I'm hoping we'll see further releases in the Llama 4 series that are smaller. I'm particularly excited to see if they produce a ~24B model, since that appears to be the sweet spot for running models on my 64GB laptop while still being able to have other applications running at the same time. Mistral Small 3.1 is a 24B model and is absolutely superb.
(Fleshed this comment out a bit on my blog: https://simonwillison.net/2025/Apr/5/llama-4-notes/#my-hopes...)