Performance tests aren't unusual. But sometimes things get slower out of necessity. It's impossible for a test to automatically distinguish between intentional and unintentional slowdowns. At some point you have to have someone make a judgment call about updating the test to accept the new state of things. Or draw a hard line and say things are never allowed to get slower no matter what, but that can be a tough goal to maintain.
Performance tests aren't unusual. But sometimes things get slower out of necessity. It's impossible for a test to automatically distinguish between intentional and unintentional slowdowns. At some point you have to have someone make a judgment call about updating the test to accept the new state of things. Or draw a hard line and say things are never allowed to get slower no matter what, but that can be a tough goal to maintain.