Comment by kthejoker2

As someone who has implemented plenty of ML-on-IIoT solutions across clients of various sizes, industries, and maturities ..

I'll just say it's unimpressive low hanging fruit to write an article about any wildly popular subject to rehash basic mental models-as-objections-to-hype like Sturgeon's Law, Maslow's hammer, Occam's Razor, and YAGNI.

ML is objectively great at finding signal in noise. When you know what signal you're looking for and how much it's worth, ML is very economical - at certain scales, it's the only sustainable option.

Here are 6 steps to evaluating ML success. The most important point is after each step, if you don't have the economics right, stop what you're doing and go use your money for something else.

1) Choose a pattern recognition objective with a financial benefit Quantify your objective over a fixed timeframe. Focus on how the objective will be achieved. "If we can reduce MTTR by 35% and reduce our fixed maintenance cycle time by 15%, we will see $11.7m gains in net revenue over the next 5 years by reducing our maintenance crew contracts."

2) Draw a clear picture of the data you have and its relationship to the objective. Identify the target variable(s). Assess the quality and granularity of your labeled data. Identify relationships between your data. Declare a hypothesis on the marginal relationship between the model loss function and the objective. Eg: "If this model is 90% accurate we expect to see a 6% lift in upselling / increase maintenance cycles by 14%, etc."

3) Bring a devil's advocate into the room and let them try to tear the idea apart. See if you can piece it back together. Compare your theoretical signal to whatever heuristics you have today. Why is it different? Can you just automate those?

4) Do a 6 week pilot to find the signal in the noise to prove your hypothesis. Do not worry about "cloud scale data platforms." Do not do MLOps. Do not build pipelines. Move heaven and earth to get data out of source systems quickly, by hand if you need to - copy and paste is also data engineering, have a domain SME and a data scientist joined at the hip for avoiding rabbit holes. Use a Kaggle-style holdout set for final model evaluation.

5) Do a field pilot, with your devil's advocate as the judge. A/B testing is usually best - think John Henry vs. the steam engine. Again, move heaven and earth to test your hypothesis in a real world setting. Bathe in uncertainty and confidence.

6) Estimate the cost to create and operate the ML pipeline needed against your expected benefits. Is it justified? Build it; otherwise, either look for ways to augment its value or kill it.

Most ML goes astray at steps 1 or 2. But a lot of good ML solutions are missed because the "elegant scam" skips most of these steps altogether.