Comment by collinmcnulty

My first big career move was similar. Employer had all these complicated ML models that played great at conferences, but operators who had to use them said they were inaccurate and indecipherable. I asked the operators what they used to sort real vs false alarms, and the answers were very simple mathematical relationships. Ripped out the ML, coded up the operators versions with some slightly tuned up statistics and got dramatically better results.