Comment by Haeuserschlucht
9 months ago
That's not an A/B test, it's a choice. A/B means, each one is displayed to different customers as the only option to then see who wanted what.
In this case, a choice is more insightful than A/B.
What I find more interesting is that people give a shit about shower heads.
I think it is an A/B test, as you described, based on this part:
> We created a secret landing page. The product and design were identical. The only difference? One was labeled “Made in Asia” and priced at $129. The other, “Made in the USA,” at $239.
It's odd that they changed the text AND nearly doubled the price. They seem to attribute the conversion rate dropping to the text change, though.
The post is unclear on this. I'm not sure where the "secret landing page" came into play here. They even show a screenshot with both options visible.
And they also say: "The visitors were given the choice to either buy the Made in USA or the Made in Asia version."
A/B would be randomly showing either ONLY the USA with higher price to some people and ONLY the cheaper Asian one to others.
However, even that isn't apples to apples as the price is obviously different.
I'm assuming they mean "in aggregate, visitors were shown both options" rather than "each visitor was shown both options".
2 replies →
One of the first lessons people learn in many fields and walks of life: People's declared preferences are often radically opposed to their revealed preferences.
"Revealed preferences" should really be called "revealed behaviors".
"Sure this alcoholic says he wants to quit booze, but when we put a discount liquor shelf at the checkout where he shops for groceries, he started buying whiskey again. What a hypocrite!"
I don’t want to eat a ton of junk food. That’s why I buy little of it.
But if it’s around, yeah, I’ll eat tons of it.
“Revealed preference! Actually you wanted to eat a ton of junk food! You were lying to yourself!” No, I actually don’t want to, in fucking fact.
Is there a way to successfully and repeatedly screen for this?
As far as I'm aware the only thing that works is very skilled use of focus groups and market research, although something similar exists in the world of politics too. At the end of the day though it's tough, people are often unaware of the disconnect in themselves, we are rationalizing machines after all. In areas where feedback from some kind of market is lacking, it's a hard problem to solve.
Economists deal with this a lot, and being economists they create models or modify existing models to account for this gap. That certainly seems to work, to an extent, but only when applied to populations rather than individuals.
On a less academic note this is a major problem for science that relies on surveys, because even when anonymous people have an awareness that they're "speaking" to someone and being "judged" in some way. People, even in that moment alone with a survey, want to reinforce the image they have of themselves. When asked by a survey, "What do you want from a new newspaper?" Very few people respond, "Celebrities, scandals, and lewd pictures." People often skew to asking for thoughtful, long-form, in-depth reporting.
BUT... then they buy tabloids and click on bait, and they don't read the complex, nuanced, long-form stuff. If they aren't even consciously aware of that, short of getting them in a behavioral lab, how do you tease that out of them? Well structuring survey questions with redundant questions phrased differently can help, you can get a sense of someone's overall "sentiment" for example, but it's still limited for the reasons described above.
A few interesting examples from :
https://scholar.harvard.edu/files/laibson/files/how_are_pref...
https://pubmed.ncbi.nlm.nih.gov/36434890/
https://www.sciencedirect.com/science/article/abs/pii/S00472...
1 reply →
Judge people on what they do not what they say. Or did you mean something else?
2 replies →
It is a whole field of research, e.g. in the food industry or in polling. You have to get creative in your setup.
Keywords to search for are "implicit" vs. "explicit" preferences
If there were, it would replace jury selection.
Omniscience?
1 reply →
I've been amazed recently by 2 shower heads. My grandmas shower has VERY low water output from the lower spigot, but flowing through the showerhead you would never know it. They replaced a showerhead at the gym recently and the flow/coverage is amazing. I want one like that in my next house, or maybe this one lol.
It consistently amazes me how much other people seem to not give a shit about appliances/tools. Everything seems to be low-lifespan garbage or have a fatal flaw or be over-featured and inconvenient to use or have inexplicable design. It seems like even the businesses who are actually trying still have issues - like all generational knowledge is gone.
Ask people with long hair how they feel about showerheads. It reveals a subgroup of users who tend to be more attentive out of self-interest.