Comment by NitpickLawyer
10 hours ago
Training cost is directly impacted by inference cost nowadays. Most of the gains come from RL these days, and that is highly dependant on inference (~7:1 inference:training in units of compute). That's mainly because you want many roll-outs for each training scenario.
Of course inference efficiency is dictated by model architecture, size, etc. You can still guesstimate some of those and have an idea about cost/serve at several size tiers.
That isn't really relevant to GP's point. We still don't know what training costs because we don't know how much RL is done.