Comment by kamranjon

Have you thought about running a second tier of the Pelican benchmark where you see which model makes best pelican on lowest or no reasoning settings? I think that'd be pretty interesting and might help highlight which models have a baseline capability - even within those I'd imagine the token usage would vary wildly and might give some indications on verbosity as well.