Comment by KekDemaga
7 years ago
How would you control for that in a real sense? Often "we controlled for" means we made some pretty big logical leaps that we justified with statistical handwaving in my experience.
7 years ago
How would you control for that in a real sense? Often "we controlled for" means we made some pretty big logical leaps that we justified with statistical handwaving in my experience.
These considerations are usually elaborated at length in the studies. And usually take more things into account that what people think about in the first 5-50 minutes after hearing about the study.
That said a lot of studies are crap, maybe the conceptual framework is sound, but they botched the mathematical parts (the stats handwaving), or the sample size is too small, so even if the math is beautiful, the study is meaningless.
Now, for these data from the real world studies the controls are the differences between individual games/movies/albums. Because there are games that take more time to upload (because they take more time to crack, or the game publisher aggressively started to shut down torrent trackers). For movies and albums I don't know what kind of controls they came up with, but they can look at IP addresses of big torrent swarms and asking film studios for marketing spending data and then later cinemas for seats sold data, and try to look at the signals. (Did downloads with a lot of IP addresses from a state had an affect on cinema tickets sold in that state? Is there a per movie difference? Is there a common cause variable - they can try to control with DVD/Blueray/video-on-demand sales.)