Comment by me_vinayakakv
6 days ago
What is the way out in such cases?
I've hit this with gemini-2.0-flash and changing the prompt ever so slightly seems to make things work, just to break it at other input.
6 days ago
What is the way out in such cases?
I've hit this with gemini-2.0-flash and changing the prompt ever so slightly seems to make things work, just to break it at other input.
Finetuning is deep learning training. It's pretty difficult to get right.
Andrej's 2019 blog laments on some of the reasons why it is hard and I can relate to a lot of this - https://karpathy.github.io/2019/04/25/recipe
The biggest mistake I see people making is this quote from the blog: "a 'fast and furious' approach to training neural networks does not work and only leads to suffering"
I'll probably write more about it in a few months...