Photo by Lukas Blazek on Unsplash
After my wife switched to a plant-based diet, I felt inspired to do more research about how such a diet affects long-term health. This proved challenging: For every researcher out there extolling the benefits of such a diet, there’s another that speaks out against it.
That’s not the first time I’ve made such a discovery: It seems that for almost every study indicating the truth of one theory, there’s another proving the opposite.
Which leads us to ask: How do we know which science is right?
Interestingly, over a decade ago Stanford University professor John Ioannidis published a paper entitled, Why Most Published Research Findings Are False.
“The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field,” wrote Ioannidis. (In other words, if something sounds too outrageous to be true, there’s a great chance that it is.)
Ioannidis’s conclusion? “Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true.”
The situation in the scientific community is influenced by a myriad of factors. Scientists are under pressure to make exciting discoveries and grab headlines for scientific journals. The more attention-grabbing their research, the higher their chances of receiving more funding–in a world of increasing competition.
“In the 1950s…the entire club of scientists numbered a few hundred thousand,” writes The Economist. “As their ranks have swelled, to 6 [million] -7 [million] active researchers on the latest reckoning, scientists have lost their taste for self-policing and quality control. The obligation to ‘publish or perish’ has come to rule over academic life. Competition for jobs is cut-throat…Nowadays verification (the replication of other people’s results) does little to advance a researcher’s career. And without verification, dubious findings live on to mislead.”
This would help explain why we see headlines like, A Glass Of Red Wine Is The Equivalent To An Hour At The Gym, Says New Study. (“That…doesn’t even sound like science,” quipped comedian John Oliver. “It’s more like something your sassy aunt would wear on a tee-shirt.”)
Of course, there’s lots of useful scientific research out there–and the right data can help us disprove our own biased conceptions and lead to superior learning, with major benefits. (If you’re interested, I’ve discovered lots of benefits to increasing the amount of plant-based foods to my diet. Still, I’m not ready to give up meat and dairy just yet…although I support my wife’s decision.)
In the end, it may be best to look at scientific studies with a skeptical eye–especially if the research sounds too good to be true.
Because a little further testing will probably reveal that it is.
Enjoy this post? Check out my book, EQ Applied, which uses fascinating research and compelling stories to illustrate what emotional intelligence looks like in everyday life.
A version of this article originally appeared on Inc.com.