Comment by iforgotpassword
2 years ago
> The worst thing for me is how the Amazon search algorithm seems to want to show you everything but the item you searched for.
Hey boss I made the site better! Through rigorous A/B testing I could figure out a way to tweak our search algorithm so people spend much more time on our site! It seems they now really enjoy browsing for products!
Ok but seriously, I have witnessed A/B testing go wrong in the past so I'm biased to blame everything on it. I wouldn't think this particular thing happened though. :)
What I could imagine is that they measure number of items bought or money spent, but even then if eg you don't also track how much of these people return stuff later you still might draw the wrong conclusions. Figuring out that a user is less likely to use your site six months down the line due to building frustration is even harder.
Amazon definitely tracks returns in their A/B tests, along with impact on long-term projections of customer value. What they also track is ad and sponsored products revenue. The sad truth with most Internet products is that advertisers are really good customers. They will pay you a lot of money with huge margins, and it's really hard for a business to say no to that.
My best guess is the algorithm has been tweaked to return exact results maybe 1/10 or 1/20 times, like a slot machine with the psychological manipulation and “reward centre activation” that comes with it.
> rigorous A/B testing
Also known as unethical, non-consensual human experimentation for profit maximization purposes.
I don't expect you to back off of your take, but you should really consider how and why you came to this conclusion.
If I put two different marketing messages on two different billboards to test whether one is more effective than the other, is that unethical non-consensual experimentation? If not, how is it different from A/B testing?
> you should really consider how and why you came to this conclusion.
Why? Is it false?
I get that reaction a lot. People have directly called me unhinged. I don't really mind.
Many of my ideas I developed by discussing this stuff with people on this site. I guess not every idea is socially acceptable. That's fine. I still want to express them.
> is that unethical non-consensual experimentation?
Yes. It's really not any different than some published psychology experiment. In fact it's much larger in scale, has much uglier interests behind it, has proprietary and unpublished results. Social sciences wish they could get away with shit like this!
Only reason it's "legitimate" is everyone depends on it to make their millions. Because money excuses everything. Just like unending amounts of first party malware corporations ship to users on a daily basis. We used to recognize that stuff as the malware it is: adware, spyware. But then corporations started doing the same thing and suddenly it's "legitimate" because they put some clause in some terms nobody reads.
In strict terms yes, if you didn't get informed consent from your test subjects that would be unethical.
Research has a lot of policies and systems set up to ensure that if your testing involves people, you must get informed consent from the persons before even trying to do the test, and it's really not hard to imagine why this is a stringent standard -- it's very easy to miss how "simple tests" can and often are adverse to those participating in the test or have unintended consequences that the researchers didn't accommodate for, regardless of the reason they did not.
Ads are often portrayed as harmless but, like, there's a reason there are restrictions on advertising for certain highly addictive products and regulations against false or misleading advertising, or certain tactics aren't allowed.
3 replies →
I wouldn't make as strong a claim as the parent comment myself, but someone pointed out to me recently that A/B testing is really similar to cold reading. Is it morally equivalent to suggest to someone, in bad faith, that you're able to deliver messages from their dead loved ones, and to perform an A/B test of switching around menu items or change up some language to try and get fewer people to abandon their carts?
I lean towards "no" but I have trouble either accepting or rejecting the proposition. It's hard for me to say that A/B testing is done in bad faith, but it's also hard for me to say it's entirely unmanipulative, either.
We do non-consensual human experimentation all the time. Whenever you try a new outfit you're doing it.
It's an extremely broad category that contains good things (installing cycle lanes to see if they encourage cycling), neutral things (making both flower mugs and wave mugs and seeing which sells) and bad things (use your imagination).
Narrowly true, but what's the difference between this and a diner trying out new blueberry pancake recipes?
It's unethical for a diner to try out new recipes. Per OpenAI policy you need to ask consent before trying out new recipes.
I hate what advertising has done to the modern web just as much as anyone, but this strikes me as hyperbole. Does making this sort of claim not make you… tired? What’s the point of arguing like this?
Nazi Germany and the Tuskegee Experiment are examples of “unethical, non-consensual human experimentation”. A/B testing features of software usually doesn’t make the same list.
> Does making this sort of claim not make you… tired?
Nope. Being used as an unwitting guinea pig for a trillion dollar corporation sure as hell makes me tired though. It's extremely tiresome and demoralizing, knowing that just so much as browsing their website contributes to their profits.
Basically we should have to consent for them to profit off of us in any way.
Godwin's Law.*
If you're from a certain background it's exactly as described. In academia, frankly probably everywhere but tech, experiments as a term of art require consent when they involve humans.
* n.b. you really should have left it out, it was a good post through "hyperbole", got close-minded in the next sentence, then just sort of blew the hatch doors off. Sometimes we just don't know something someone else knows. Not understanding someone else doesn't require they have a psychological condition, much less one worth noting.
15 replies →
That's a bit of a false dichotomy. It doesn't have to be either Nazism or totally chill.
You know the Nazis had pieces of flair that they made the Jews wear.
You are consenting to the experimentation by accessing the software. It’s covered by the terms and conditions.
Terms and conditions? It's hard to tell when people are joking. https://www.youtube.com/watch?v=AhsyZ5V1pok
> terms and conditions
Nobody reads that stuff.