Comment by naasking
4 days ago
> We're normally data-limited.
This is a common sentiment but is probably not entirely true. A great example is cosmology. Yes, more data would make some work easier, but astrophysicists and cosmologists have shown that you can gather and combine existing data and look at it in novel new ways to produce unexpected results, like place bounds that can include/exclude various theories.
I think a philosophy that encourages more analysis rather than sitting back on our laurels with an excuse that we need more data is good, as long as it's done transparently and honestly.
This depends on what you are trying to figure out.
If you are talking about cosmology? Yea, you can look at existing data in new ways, cause you probably have enough data to do that safely.
If you are looking at human psychology? Looking at existing data in new ways is essentially p-hacking. And you probably won’t ever have enough data to define a “universal theory of the human mind”.
> If you are looking at human psychology? Looking at existing data in new ways is essentially p-hacking.
I agree that the type of analysis is important, as is the type and quality of the data you're analyzing. You can p-hack in cosmology too, but it's not a quality argument there either.
> And you probably won’t ever have enough data to define a “universal theory of the human mind”.
I think you're underestimating human ability to generalize principles from even small amounts of data [1]. Regardless, my point was more specifically that we could use existing data to generate constraints to exclude certain theories of mind, which has definitely happened.
[1] https://en.wikipedia.org/wiki/Predictive_coding
I suspect you didn't read some parts of my comment. I didn't say everyone in the world is always data-limited, I said we normally are where I work. I didn't recommend "sitting back on our laurels." I made very specific recommendations.
The qualifier "normally" already covers "not entirely true". Of course it's not entirely true. It's mostly true for us now. (In fact twenty years ago we used more numerical models than we do now, because we were facing more unsolved problems where the solution was pretty well knowable just by doing more complicated calculations, but without taking more data. Back then, when people started taking lots of data, it was often a total waste of time. But right now, most of those problems seem to be solved. We're facing different problems that seem much harder to model, so we rely more on data. This stage won't be permanent either.)
It's not a sentiment, it's a reality that we have to deal with.
> It's not a sentiment, it's a reality that we have to deal with.
And I think you missed the main point of my reply: that people often think we need more data, but cleverness and ingenuity can often find a way to make meaningful progress with existing data. Obviously I can't make any definitive judgment about your specific case, but I'm skeptical of any claim that it's out of the realm of possibility that some genius like Einstein analyzed your problem could get no further than you have.
Apparently you will not be told what I'm saying.
I read your point and answered it twice. Your latest response seems to indicate that you're ignoring those responses. For example you seem to suggest that I'm "claim[ing] that it's out of the realm of possibility" for "Einstein" to make progress on our work without taking more data. But anyone can hit "parent" a few times and see what I actually claimed. I claimed "mostly" and "for us where I work". I took the time to repeat that for you. That time seems wasted now.
Perhaps you view "getting more data" as an extremely unpleasant activity, to be avoided at all costs? You may be an astronomer, for example. Or maybe you see taking more data before thinking as some kind of admission of defeat? We don't use that kind of metric. For us it's a question of the cheapest and fastest way to solve each problem.
if modeling is slower and more expensive than measuring, we measure. If not, we model. You do you.