Comment by tarruda

4 days ago

Thanks for your work, it is really an amazing small LM.

Can you share what kind of hardware is necessary to train it, and how long it took?

1 comment

tarruda

Thank you!

The Gemma3 technical report contains many details on training setup https://arxiv.org/pdf/2503.19786

This was released with the initial batch of Gemma3 so it doesn't contain the 270m details, nonetheless you'll get a good idea of what it takes to build these models.