If you missed chaper #4 you can find it here.
You probably read it on the headline of many important newspapers, “Chocolate can help you lose weight!”, “Eat chocolate to lose weight faster!” and many have fallen for this hoax. But the interesting thing about it, is that there was an actual research paper behind it.
How is that possible?
How can an actual research produce a hoax as result?
Well, this experiment was accurately designed to increase the probability that a false positive would show up. As John Bohannon, co-author of the paper, says:
"Here's a dirty little science secret: If you measure a large number of things about a small number of people, you are almost guaranteed to get a 'statistically significant' result. Our study included 18 different measurements -- weight, cholesterol, sodium, blood protein levels, sleep quality, well-being, etc. -- from 15 people. (One subject was dropped.) That study design is a recipe for false positives"
He continues: “We didn't know exactly what would pan out -- the headline could have been that chocolate improves sleep or lowers blood pressure -- but we knew our chances of getting at least one 'statistically significant' result were pretty good.”
John Bohannon knows perfectly what p-hacking is.
The p-value
P-value in science, explained in the simplest way, is the number that express the probability of your experiment result happened by chance.
Or, to be more precise, the probability that the relationship between two variables (in the case above eating chocolate and weight loss) is given by chance.
Usually a p-value of 0.05 is significant, but, as said above, if you have a small number of test subjects and a lot of variables to measure you should expect a lot of noise in your data.
P-hacking is the manipulation of the p-value in order to have the desired result and it is one of the greatest problem in science. Of course, since science is peer reviewed it would be easy to replicate those hacked experiments and prove them wrong, but considering that millions of new scientific articles are published each year a great number of papers will go unchecked and therefore some of them will result in bad science.
In this Series we focused our attention on the nature of reality, on our senses, our interpretation of them and on the way we analyse objectively reality: The Scientific Method. Finally in this article we focused instead on one problem, scientific results can be falsified intentionally or by mistake.
But why is it important for us to know all of this and how does it influence our everyday life?
How does having an accurate understanding of the world may improve our life?
We’ll answer these questions in the final chapter of this series.
Super-Duper Important Footnote
I want to thank each one of you who takes the time to read these posts, the majority of them take some time to make and it feels good when someone enjoy reading your content.
If you read up to here consider to leave a comment with your opinion on p-hacking in science and why, in your opinion, understanding the world is the most powerful for an individual.
Thanks again, I’ll see you in the next article!
Great post! It sounds like we might have similar interests. I have been looking for more people to follow! Thanks! Followed!
Thank you! I've seen from your profile you focus on hacking, as a software engineer I'd love to see more articles from you. If you look for some other science blogs I would like to suggest you @thepe , he's making some exceptional posts ;-)
Thanks! I will go check out @thepe's blog! I just joined steemit and am still looking for content to subscribe to!
I'm learning how to use the API and writing a script that finds authors based on my interests. lol I have apparently been spamming a bit much, but I have also found a lot of great authors to follow.
That should be the subject of one of your posts! I will surely enjoy reading it!
Again a nice post! A little sentence comes to my mind, which we have in Germany: "Traue keiner Statistik, die du nicht selbst gefälscht hast". I don't know, if it exists in English, but translatet it would be something like "Don't believe in a statistic which you don't counterfeited yourself". (I just read, that the sentence might originally be from Churchill, but however...)
As you explained, if you read a statistic analysis wrong (intentionally or by mistake), you are able to "prove" nearly anything. I just found a site of three professors who show once a month an "Unstatistik" (maybe it's like "anti-statistic?") and discuss the failures. (Link - German site) I don't have the time now to read it, but it might bring some interesting effects.
But anyway, we should not forget, that statistics are useful as well. And I don't want to impute someone to counterfeit their statistics intentionally!
P.S.: Thank you for advertising my blog ;-)
Congratulations @bendelgreco! You have completed some achievement on Steemit and have been rewarded with new badge(s) :
Click on any badge to view your own Board of Honor on SteemitBoard.
For more information about SteemitBoard, click here
If you no longer want to receive notifications, reply to this comment with the word
STOP