Comment by sjsdaiuasgdia

Compare the results between the current code and the proposed version. Analyze what the new version blocks that the current one doesn't and vice versa. Have logging that shows which factor(s) were applied in the actions taken. Determine if the outcomes are in line with the intended goals.

This can be accomplished in a few ways. You could accumulate real URLs and build a test set that you can run in non-prod environments prior to deploy. You could also deploy the new version alongside the current version, both watching the live data, with the current version retaining enforcement power while the new version is in log-only mode.

In the case of automated systems that might create new actions in response to live traffic, anomaly detection can be used to look for significant changes in classification and/or rate of actions, spikes in actions against specific domains, etc.