I don't think "compartmentalization" does this topic justice. Note, I'm not accusing you, since I don't know your work. My post is about the general milieu and the rhetoric around it. This "compartment" terminology seems to whitewash a dimension that has diligence on one end, and fraud on the other.
I've always worked in high-trust organizations where we depended on each other operating on the diligent side of things, making conservative choices to never wander into the murky area in between. It is horrifying to me how many people seem complacent about or even complicit in a different objective, which seems hell bent on wandering as far into the murk as one can without being caught.
When the person who has a duty of diligence starts rubber-stamping AI outputs, they're veering off into that same murk. They accidentally or wantonly trust the agent as if they have delegated their duty of care. But the AI tool has no such duty and no capacity to care.
I think this worker who has turned themselves into an outsourcing middleman needs to treat the results just like "found code" in a USB drive they found in the parking lot. Its origins and purpose are unclear. It could be flawed or obscurely inappropriate for the intended application, it could have legal entanglements, or it could even be subtly adversarial. The review task to figure this out is not simple. It is not something you do by skimming the result, or worse, asking some other AI tool to review and summarize.
The person importing such code to a project needs a different kind of diligence to try to screen it. For a lot of people, I think this review may be impossible or at least no less laborious than doing the original work themselves with the required diligence.
And, I think this importer needs to be fully liable and responsible for the outcome. But, instead, I think we're seeing frequent appeals to blame the machine and act like it is an honest mistake to let things pass because they've been rubber-stamping the imports. A lazy desire to claim credit for appearance of success, but shirk responsibility for detected failures.
Software Engineers have historically not been very liable even for errors they create by their own hands, and companies tend to operate on a “blameless” engineering culture where there is not really a formal consequence for screwing something up. Your peers may not trust you for a bit, but the trust builds back rapidly as they make their own mistakes and then empathize with your past transgressions.
And in the AI era, this attitude has extended even toward code engineers aren’t writing themselves. Everyone agrees not to protest too loudly if someone commits something wrong because tomorrow it might be you who has done so. The days of “obviously bad” slop code are largely behind us and most recent LLMs tend to put out decent code, so it is very hard to catch problems in review phases, you’d have to be very eagle eyed and that’s not scalable to every employee. It is much easier to let the code run wild for a bit and just be ready to fight fires should they come up.
Nothing wrong with compartmentalization. Some people like it, some people don’t. Just different ways to live your life.
I don't think "compartmentalization" does this topic justice. Note, I'm not accusing you, since I don't know your work. My post is about the general milieu and the rhetoric around it. This "compartment" terminology seems to whitewash a dimension that has diligence on one end, and fraud on the other.
I've always worked in high-trust organizations where we depended on each other operating on the diligent side of things, making conservative choices to never wander into the murky area in between. It is horrifying to me how many people seem complacent about or even complicit in a different objective, which seems hell bent on wandering as far into the murk as one can without being caught.
When the person who has a duty of diligence starts rubber-stamping AI outputs, they're veering off into that same murk. They accidentally or wantonly trust the agent as if they have delegated their duty of care. But the AI tool has no such duty and no capacity to care.
I think this worker who has turned themselves into an outsourcing middleman needs to treat the results just like "found code" in a USB drive they found in the parking lot. Its origins and purpose are unclear. It could be flawed or obscurely inappropriate for the intended application, it could have legal entanglements, or it could even be subtly adversarial. The review task to figure this out is not simple. It is not something you do by skimming the result, or worse, asking some other AI tool to review and summarize.
The person importing such code to a project needs a different kind of diligence to try to screen it. For a lot of people, I think this review may be impossible or at least no less laborious than doing the original work themselves with the required diligence.
And, I think this importer needs to be fully liable and responsible for the outcome. But, instead, I think we're seeing frequent appeals to blame the machine and act like it is an honest mistake to let things pass because they've been rubber-stamping the imports. A lazy desire to claim credit for appearance of success, but shirk responsibility for detected failures.
This view is a little idealistic.
Software Engineers have historically not been very liable even for errors they create by their own hands, and companies tend to operate on a “blameless” engineering culture where there is not really a formal consequence for screwing something up. Your peers may not trust you for a bit, but the trust builds back rapidly as they make their own mistakes and then empathize with your past transgressions.
And in the AI era, this attitude has extended even toward code engineers aren’t writing themselves. Everyone agrees not to protest too loudly if someone commits something wrong because tomorrow it might be you who has done so. The days of “obviously bad” slop code are largely behind us and most recent LLMs tend to put out decent code, so it is very hard to catch problems in review phases, you’d have to be very eagle eyed and that’s not scalable to every employee. It is much easier to let the code run wild for a bit and just be ready to fight fires should they come up.