Comment by makeitdouble

2 years ago

An issue could be how people choose that week.

As you point out that's a multi day commitment, and if part of the volunteers either adjust the timing of the experiments to match specific weeks (e.g. parents choosing school vacations), or adjust their schedule accordingly, what is measured becomes fundamentally different in nature to what measuring longer periods would bring.

I'm with you on how we don't have a choice regarding to the quality of the study, it's just crazy hard to get any data at scale. But we can look at it as a very flawed "best of what we can do" and not take the patterns too seriously.