not neccesarily? It depends on what the claims are. If prior art says 'this is impossible' then a study with one result that is hard to replicate is an interesting insight about the margins. If it is commonly accepted wisdom, we may need to address what that is. If it is your experiment, it doesn't mean your result is wrong but it means it should be significantly questioned. If I say its more that is on me...if I say it is what it is then the results are the results.
I think it is a growing, and problematic, misconception that science is about just reporting facts. It is about adding knowledge...soemtimes that is new facts, but more often it is theories, observations, and questions increasingly at the margins of randomness and probability as our species grows and learns.
Substack psychologist Adam Mastroianni argued that if you need advanced statistics to tease out some effect in the soft sciences, it is actually likely that the effect is bogus.
How many of those non-reproducible studies showed a robust effect, in contrast to the barely there effect?
so, measuring progress by lines of knowledge added?
speaking openly
that kinda science seems like a bullshit factory (a very lucrative one, of course)
and analogous to an llm hallucination — shallow satisfaction of constraints by enumerating a problem space irrespective of contextual appropriateness, a liability, and forgotten as soon as real value comes along
surely the best science isn't just reproducible by scientists — engineers can get the grubby mits on it, to improve (ok, shape) the circumstances of our lives, and survival
that other stuff is busywork, not "literature". they are very different things...
not neccesarily? It depends on what the claims are. If prior art says 'this is impossible' then a study with one result that is hard to replicate is an interesting insight about the margins. If it is commonly accepted wisdom, we may need to address what that is. If it is your experiment, it doesn't mean your result is wrong but it means it should be significantly questioned. If I say its more that is on me...if I say it is what it is then the results are the results.
I think it is a growing, and problematic, misconception that science is about just reporting facts. It is about adding knowledge...soemtimes that is new facts, but more often it is theories, observations, and questions increasingly at the margins of randomness and probability as our species grows and learns.
Substack psychologist Adam Mastroianni argued that if you need advanced statistics to tease out some effect in the soft sciences, it is actually likely that the effect is bogus.
How many of those non-reproducible studies showed a robust effect, in contrast to the barely there effect?
so, measuring progress by lines of knowledge added?
speaking openly
that kinda science seems like a bullshit factory (a very lucrative one, of course)
and analogous to an llm hallucination — shallow satisfaction of constraints by enumerating a problem space irrespective of contextual appropriateness, a liability, and forgotten as soon as real value comes along
surely the best science isn't just reproducible by scientists — engineers can get the grubby mits on it, to improve (ok, shape) the circumstances of our lives, and survival
that other stuff is busywork, not "literature". they are very different things...