Comment by albertzeyer
20 hours ago
I have worked both with the TensorFlow C++ API and the TensorFlow Python API. While the TF Python API is basically only a wrapper around the TF C++ API, it adds a lot of things on top, e.g. many higher-level functions you would want to use to define neural networks, etc. If you know PyTorch, think about torch.nn. Most crucially, calculating the gradients, i.e. doing backprop/autograd, was also purely implemented in Python. Even to define the gradient per each operation was done in Python. The C++ core did not know anything about this. (I'm not exactly sure how much this changed with eager mode and gradient tapes though...)
So, that makes implementing training with only the C++ API quite a big task. You first need to define all the gradients, and then implement backprop / autograd.
Some of the gradients are present in the TF C API (and thus the C++ API), but it's hit & miss which are in C and which are in Python. There was an attempt years ago to port more gradients to C, but this petered out in the way that most TF related efforts seem to do at Google.
We use the C API to generate gradients for TF-Java, and have some success training models with it. Replicating the Python bits in another language is a huge effort though that we haven't completed.