Comment by gdiamos

Finetuning is deep learning training. It's pretty difficult to get right.

Andrej's 2019 blog laments on some of the reasons why it is hard and I can relate to a lot of this - https://karpathy.github.io/2019/04/25/recipe

The biggest mistake I see people making is this quote from the blog: "a 'fast and furious' approach to training neural networks does not work and only leads to suffering"

I'll probably write more about it in a few months...