Comment by wcarss
3 days ago
Lead: "We have six weeks to ship. Questions?"
Dev: "Could we pull an export of relevant historical data and get some time to write code to safely anonymize that, and stand up a parallel production system using just the anonymized data and replicate our deploy there, so we can safely test on real-ish stuff at scale?"
Lead: "I'll think about it. In the meantime, please just build the features I asked you to. We gotta hustle on this one."
I'm not arguing with this hypothetical exchange that it's infeasible or even a bad idea to do exactly what you suggested, but attempting to justify an upfront engineering cost that isn't directly finishing the job is a difficult thing to win in most contexts.
It's very common to use identical systems but anonymised data shipped back to test environments in such cases. There are certain test card numbers that always fail or always succeed against otherwise-real infrastructure on the card provider's side.
Absolutely, I agree that it's a useful pattern. I've personally typed 4111 1111 1111 1111 into a stripe form more times than I want to even think about.
My point above was that it's not necessarily easy to convince the operators of a business that it's a justifiable engineering expense to set up a new "prodlike but with anonymized data" environment from scratch, because it's not a trivial thing to make and maintain.
I do think it's pretty easy to convince operators of a business to adopt the other strategy suggested in a sibling thread: run a dry mode parallel code path, verify its results, and cut over when you have confidence. This shouldn't really be an alternative to a test environment, but they can both achieve similar stuff.
> I do think it's pretty easy to convince operators of a business to adopt the other strategy suggested in a sibling thread: run a dry mode parallel code path, verify its results, and cut over when you have confidence. This shouldn't really be an alternative to a test environment, but they can both achieve similar stuff.
I agree - it's a nice low-risk way of doing things.
1 reply →