Comment by imtringued

This is why easy "out of the box" continual learning is absolutely essential in practice. It's not like the LLM is incapable of solving tasks, it simply wasn't trained for your specific one. There are optimizers like DSPy that let you validate against a test dataset to increase reliability at the expense of generality.