← Back to context

Comment by thatguysaguy

1 year ago

I have worked on a team which trains foundation models

And no one’s used anything other than Python? The core of Tensorflow uses C++

  • Many parts of tensorflow required Python- at least when I worked there a few years ago, it was nearly impossible to compile XLA into a saved model and execute it from pure C++ code.

  • The claim was that Python is only used for prototyping at Google, not that people are writing the AI frameworks themselves in pure Python.

    JAX is obviously implemented in C++, but the scientists running the training runs which cost millions of dollars are writing lots of python.