Comment by aidenn0
2 hours ago
My impression is that AI didn't replace static code in this place; it replaced a person, who (hopefully) would have been suspicious about sending an account recovery code for e.g. "obamawhitehouse" to e.g. "bscurtu.alfamm.ro@gmail.com"
You're giving a lot of credit to the human alternative, especially considering that the attacker only needs to find one lazy human.
Still makes this exponentially worse, no? It works every time and it's automated so scales up as quickly as you're able to request it.
Come on, this attack vector would have been flagged by at least one person and you won’t then have multiple accounts hacked because of it. AI reacts fairly predictably to a single attack vector and don’t learn unless it gets flagged and then taught.
And even if a human didn’t catch it in one case, they will frequently. Giving AI access to the same tools humans use without any oversight mechanism just amplifies the harm and carelessness possible by one person.