What is truth? P-hacking in scientific papers

avatar
(Edited)

Not all data is good data, and not all statistically significant results are helpful

A lot of people think that truth can be found by trusting the scientific literature as the source of truth. The hidden truth or open lie, is that statistics can be gamed. In a scientific paper there is a way to make the statistics say whatever the writer of the paper wants it to say, and one was of doing this is p-value hacking. By scientific convention, if a p-values are less than 0.05, they are labeled statistically significant by this arbitrary convention. Once labeled statistically significant, it becomes scientifically true and worthy of publication.

But with p-hacking, any paper could be produced by simply searching through the data to find patterns which produce the desired statistical significance. Cherry picking, collecting data until P<0.05, then stop. Now you can say you discovered a new "scientific truth", because something statistically significant was found, even if it's of low scientific value.

There are many mice studies, many bizarre studies, which produce some statistically significant result from the data. Nothing comes of it, but it satisfies the publish or perish condition that the writers of the papers are living under. Is any of this truth? It could be statistically significant, but is it truth?

Reference
https://embassy.science/wiki/Theme:6b584d4e-2c9d-4e27-b370-5fbdb983ab46



0
0
0.000
0 comments