Comment by flowerthoughts

I would much rather that the model learns natural languages and knows how to run base64(1) to do decoding. Parameters should be considered precious so we can get the model sizes down from these absurd levels.

I'm a big fan of the Mixture of Experts approach, and having agents attached to some of those experts would be a great step forward. Say an expert that knows how to run common shell scripts, and its parameters are only used when it realizes a shell script would solve the problem.