Comment by readyplayeremma
2 years ago
> B., a senior officer who used Lavender, echoed to +972 and Local Call that in the current war, officers were not required to independently review the AI system’s assessments, in order to save time and enable the mass production of human targets without hindrances.
> “Everything was statistical, everything was neat — it was very dry,” B. said. He noted that this lack of supervision was permitted despite internal checks showing that Lavender’s calculations were considered accurate only 90 percent of the time; in other words, it was known in advance that 10 percent of the human targets slated for assassination were not members of the Hamas military wing at all.```
So, there was no human sign-off. I guess the policy itself was ordered by someone, but all the ongoing targets that were selected for assassination were solely authorized by the AI system's predictions.
This sentence is horrifically dystopian... "in order to save time and enable the mass production of human targets without hindrances"
Hm OK, I read this a bit differently. I read these sections:
> One source stated that human personnel often served only as a “rubber stamp” for the machine’s decisions, adding that, normally, they would personally devote only about “20 seconds” to each target before authorizing a bombing — just to make sure the Lavender-marked target is male.
> According to the sources, the army knew that the minimal human supervision in place would not discover these faults.
I took this to mean that a human did press the "approve" button on the computer's recommendation. Though they make clear they were basically "rubber stamping" the machine recommendation.
But to my point:
> “There was no ‘zero-error’ policy. Mistakes were treated statistically,” said a source who used Lavender.
What is the "zero-error" alternative approach for dropping bombs in a war, or firing rockets for that matter? I don't understand the implicit comparison between this approach to targeting and a hypothetical approach that allows war to be waged without any innocents dying or buildings being destroyed. This system should be compared to whatever the real alternative is when it comes to target selection. Again I know nothing about military strategy, I'm hoping someone with more experience will speak up.
To use an analogy: if we are talking about self-driving cars, the rates of collision or death should be compared the rates of collision or death in cars driven by humans. Comparing against some imaginary scenario where cars have no collisions and cause no deaths doesn't make sense.
> What is the "zero-error" alternative approach for dropping bombs in a war, or firing rockets for that matter?
Honestly, I'm not sure. Obviously humans make errors of all sorts as well, and even make intentionally unethical decisions.
I think the horror of this situation is that it makes war easier to wage. Accepting that all war has costs measured in blood, we should want less war. However, those in control of military forces always have incentive to wage war, so removing friction from the process is dangerous.
Off-topic of AI, but on-topic of your question:
The actual alternative to unleashing AI assassination is not human-selected targets, but not waging war. It isn't necessary to destroy Hamas with violence, it would have worked better to give Palestinians dignity and self-determination long ago. That can still work, although until it does Hamas will continue to be a problem. But as I said, war is useful for the political leaders of Israel, so they stoked and fed the flames for decades to maintain an excuse for the war machine.
Since you went off topic. If Palestinians only wanted dignity and self-determination this conflict would have been resolved a long time ago. Palestinians, broadly speaking, want Israel removed from the map. This is why they're chanting "from the river to the sea" which happens to include the area Israel is situated in.
During the Oslo peace process, when Israel was trying to address this in the way you propose, Hamas launched a suicide bombing campaign against Israeli civilians:
https://en.wikipedia.org/wiki/List_of_Palestinian_suicide_at...
https://en.wikipedia.org/wiki/Oslo_Accords
You can be critical of everything Israel does, in this war or ever - fine. But the Palestinians have no other accepted settlement other than shipping ~8 million Jews to Europe or killing them.
The people who suddenly developed this simplistic understanding of occupation/resistance/occupier have no idea what they're talking about. Often quite literally in the sense they don't even understand the meaning of what they're saying, not to mention the history of Israel or the middle east. EDIT: I realize this last statement can feel offensive but this is still my take based on two decades of interactions with a fairly random sample of people trying to explain wth is going on in this tiny piece of the middle east. The complexity of the situation doesn't yield itself to simplistic narratives (from neither side really, though my statement refers to one of those narratives the Israeli side simplistic narrative is also insufficient/inaccurate).
25 replies →
>It isn't necessary to destroy Hamas with violence
It isn't possible to destroy Hamas with violence, or apartheid for that matter. Israel has created hatred towards themselves that will last for generations, even if they could kill every last Hamas member, they've made damn sure that a subset of Palestinian (if not broader) youth will reorganize a militia and the cycle of violence will go on.
4 replies →
It can only work if a majority of Palestinians want to coexist with Israel. That hasn't been the case for the most part since 1948.
Eh, some people actually have different visions for the world. They'll elect people who are abhorrent to western liberal values over and over again. I don't know what a new election in Gaza would yield, but I don't think it can be a given that giving X group dignity and self-determination will necessarily tilt them toward western liberal outcomes.
8 replies →
Palestinians were given opportunities for self-determination in 1948, 2000 (Camp David), 2008, and 2006 in Gaza (blockaded by Egypt because of Hamas elected to run Gaza). In 1948, they along with 5 invading Arab countries tried to destroy Israel, resulting in their own destruction of their Arab state. In 2000, Arafat turned down a peace agreement with Bill Clinton starting terrorism that resulted in 3000 Palestinian and 1000 Jewish and Israeli Arab deaths, in 2008 Abbas turned down a peace agreement.
After 10/7 almost every Israeli knows that the Palestinians are not interested in their own state.
Of the 32,000 Hamas stated deaths, 13,000 are terrorists, thus resulting in a far lower civilian-to-combatant death ratio than in other urban conflicts such as Mosul.
The lesson learned with Japan in Germany in WW II is that total military defeat is necessary. The AI technology enables the targeting of all terrorists, not only senior-level terrorists as before, resulting in a quicker end to the conflict than otherwise and thus resulting in fewer civilian deaths.
As we know these terrorists hide among civilians including in and under hospitals, making these legitimate targets. The high number of civilian deaths occur from the terrorists hiding among civilians.
19 replies →
>> Comparing against some imaginary scenario where cars have no collisions and cause no deaths doesn't make sense.
That's not the whole story. For example, we ban certain kinds of weapons -cluster munitions, chemical weapons, biological weapons, ideally we'd ban bloody mines- not because they kill too many people compared to "conventional" weapons (they don't) but because they are considered especially ... well, wrong, in the moral sense.
So maybe we decide that being killed by a machine, that decides you're a target and pulls the trigger autonomously is especially morally wrong and we don't accept it.
For clarity, cluster munitions and mines are banned because they kill indiscriminately and pose cleanup hazards.
Chemical and biological weapons are banned because, like nukes, escalation of their use results in a scorched earth scenario.
7 replies →
The difference is between inaccuracy of a weapon hitting a target and inaccuracy of target selection in the first place.
Remember the scene in Men In Black where the recruita do target practice? They were all accurate at hitting what they shot at but only Will Smith's character was accurate at selecting a target. This AI chooses targets; it does not fire weapons.
Haha having recently rewatched MIB with my daughter after ~15 years, I don't think Will Smith correctly selected the target... :'D
7 replies →
"Just following orders" huh? I can't believe I'm being offered Will Smith analogies as apologia for an actual genocide. This is one of the most of awful (in all senses quality, content, intention, execution) posts I've ever read in my entire life.
1 reply →
I think 90 percent accuracy in this case means 10 percent of "suggested terrorists" were overturned with detailed human review. There's no way the Israelis were actually able to reliably question the Gazans about whether they really were terrorists.
So the issue isn't that there's errors, it's that the army knows there are errors and expect humans to pick them out in 20 seconds- which they know realistically won't happen. The human only has two realistic choices- approve every target, or disapprove every target (which gets you reassigned to another role).
It's the classic statistics case of two medical diagnostics for an underlying value that isn't directly observable.
> > “There was no ‘zero-error’ policy. Mistakes were treated statistically,” said a source who used Lavender. > > What is the "zero-error" alternative approach for dropping bombs in a war, or firing rockets for that matter? I don't understand the implicit comparison between this approach to targeting and a hypothetical approach that allows war to be waged without any innocents dying or buildings being destroyed. This system should be compared to whatever the real alternative is when it comes to target selection.
I think you've misunderstood the "zero-error" statement. It's not saying "there must be zero errors", rather that "errors don't exist - only some level of collateral damage". Hence the follow up about things being viewed statistically.
They view it in the same way that you suggest they should - that there will always be deaths and the questions is whether the system leads to more or less of them.
Personally I view that as a very utilitarian argument when applied to a machine of war. It embeds the concept that some loss of innocent life is acceptable.
We also have to be open to the possibility that Israel is committing a genocide and the goal is to kill as many Palestinians as possible and terrorize the rest. That the AI system’s main purpose isn’t to be accurate in selecting target, but rather to manufacture a reason to kill more Palestinians than a human ever could. Another function could be to remove accountability from a targeting officer. Zero-error is never really a desired feature, in fact zero-error would be a bug, as it would prevent the genocide being conducted efficiently.
What we may be witnessing is the first information age level genocide, where the killing is done at the behest of a statistical function with near infinite computing power.
This is an obviously wrong analysis. Killing the most people possible would require no AI in this situation.
1 reply →
And incredibly inefficient genocide. Why is it that modern discourse has become so polarized that criticism has to make the worst possible accusations?
2 replies →
Later in the article they talk about how they specifically approved up to 15-20 civilians to die with those marked individuals and would bomb their homes as a first option.
I’m disgusted by this, I don’t care anymore what happened in October, this needs to stop. Israel government cannot be trusted to run this war, it’s turned into genocide and we’re all complicit letting them do it and supporting them. I can’t believe people actually support this, it’s clear they’ve forgotten Palestinians are people.
Israeli officials are constantly being asked "how many dead palestinians is too many" in this conflict, and the answer has explicitly been "there is no such thing" way too many times. There is no upper limit on how many people can be killed to further their goals.
The most upsetting(for me) thing is reports of all the kids killed by snipers and just in general, as a father I cannot imagine losing my child to this.
https://www.theguardian.com/world/2024/apr/02/gaza-palestini...
https://www.theguardian.com/commentisfree/2024/mar/23/israel...
7 replies →
Yes its genocide.
In 1999 Yugoslavia killed ~12 thousand Albanians and displaced ~85 thousand more. Bill Clintons secretary of defense had no problem calling that genocide: "The appalling accounts of mass killing in Kosovo and the pictures of refugees fleeing Serb oppression for their lives makes it clear that this is a fight for justice over genocide.". This led NATO to drop bombs on Yugoslavia [0].
In this conflict Israel has killed ~31 thousand Palestinians and displaced ~2.3 million more [1]. And now we sell them jet planes [2].
[0] https://en.wikipedia.org/wiki/NATO_bombing_of_Yugoslavia
[1] https://en.wikipedia.org/wiki/Israeli_invasion_of_the_Gaza_S...
[2] https://www.msn.com/en-us/news/politics/biden-administration...
[flagged]
>>It's rather unfashionable these days to bring up the fact that Hamas purposely disguises themselves as civilians and operates almost exclusively from civlian buildings, and makes sure their compounds aren't separated from civilian infrastructure.
And yet, somehow I still feel like the answer to this problem still isn't "authorizing up to 15-20 civilian deaths for every enemy militant killed" as mentioned in the article.
12 replies →
I imagine you get to tune the probability window of "person is >90% likely a Hamas terrorist" and choose how many innocent people you kill. Who set the window?
"Hamas terrorist" criteria: a male of fighting age, give higher weight to those congregating with others of fighting age. Basically take out a generation of Palestinian men and you're all set. Lovely.
>This sentence is horrifically dystopian... "in order to save time and enable the mass production of human targets without hindrances"
Reminds me of similar industrial thinking of a certain previous fascist government.
A war that would only kill 10% civilians would be a massive improvement over any recent conflict.
Well, that was a 10% failure rate supposedly on selecting the primary target of the attack.
The attack itself was allowed to have a 15x to 100x number of civilians killed depending on the supposed importance of the target.
I imagine that's very WW1-y and is a sign that the military leadership is just sending soldiers to die without care
Isn’t it close to Russian-Ukrainian war ratio?
12 replies →
For every militant they correctly identify (90% of the time, they'd have us believe) and kill, they also kill dozens of innocents. This doesn't give them pause; on the contrary the Israeli public revels in the carnage and bring out lawn chairs to watch. It's genocide.
1 reply →
>Basically take out a generation of Palestinian men and you're all set.
Now that we've established that this is horrific, please turn a small portion of your attention to American predictive policing systems (digital and not) and the circumstances that lead to mass incarceration (including the War on Drugs).
...and then target them at home along with their entire family.
But you see, if you kill just them, then their family would very likely get radicalized because of that, and then you'd have to kill them too, only some time later so it's just more efficient to do it in one fell swoop while you have the chance.
Of course it's perfectly ethical, why do you ask?
2 replies →
I don’t like Lavender. I think humans should always be in the loop. I’d like to see more care by analysts for kill orders.
That said, any organization might do something if it’s 90% accurate. Assuming it even is (doubt it), I think any fair evaluation of such a technology must ask:
What is the accuracy of inexperienced humans in the same position who are rushing through the review during a blitz invasion? If they have battle experience, what about them, too? (I’m assuming most won’t.)
Is the system better than those humans or worse? How often?
Do the strengths and weaknesses of the system allow confidence scores on predictions to know which need more review? Can we also increase reviews when the number of deaths will be high?
That’s how I’d start a review of this tech. If anyone is building military AI, I also ask that you please include methods to highlight likely corner cases or high-stakes situations. Then, someone’s human instincts might kick in where they spot and resolve a problem even in the heat of war.
Lavender AI in it's current form is not a technology that should be considered at all for anything close to this. It's incompetent.
Basically AI is being used as a wheel-of-death and nothing more.
Anyone believing differently in my opinion is both delusional, and complicit
It is very clear to me that that is a sentence reflecting the editorial interpretation of the paper rather than a direct quote. You might agree with the interpretation - I think I might - but that is very different from this specific sentiment being something Israeli leadership are openly saying.
90% is a BS number . Computed basis what ? What is the baseline how did they benchmark . Is there any data whatsoever to back this claim ?
They just spout a high number that is not 100% (clearly civilians are being killed publicly undeniably ) claiming 100% would be too obviously ridiculous.
More than half of 32,000+ (more under the rubble) killed are woman and children, Hamas is still quite able to fight, hardly any hostages has been recovered .
Israel labels any sort of civilian organization as hamas including journalists, medical and aid staff. 200 UN staff and 100 journalists are dead so far . Israel’s argument is UNWRA terrorist aiding and journalists were also secretly Hamas and doing non journalistic stuff when killed so they include them in legitimate targets .
If you consider everyone is Hamas unless otherwise proven then 90% is possible .
There is no realistic way an algorithm was designed factoring in the level of destruction of infrastructure never seen in any real world data and also benchmarked accurately.
[dead]