[sciborg2]

Comment from Richard Horton in Lancet:

http://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736%2815%2960696-1.pdf

The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. As one participant put it, “poor methods get results”

Can bad scientific practices be fixed? Part of the problem is that no-one is incentivised to be right. Instead, scientists are incentivised to be productive
and innovative. Would a Hippocratic Oath for science help? Certainly don’t add more layers of research redtape. Instead of changing incentives, perhaps one could remove incentives altogether. Or insist on replicability statements in grant applications and research papers. Or emphasise collaboration, not competition. Or insist on preregistration of protocols. Or reward better pre and post publication peer review. Or improve research training
and mentorship. Or implement the recommendations from our Series on increasing research value, published last year. One of the most convincing proposals came from outside the biomedical community. Tony Weidberg is a Professor of Particle Physics at Oxford. Following several high-profi le errors, the particle physics community now invests great eff ort into intensive checking and rechecking of data prior to publication. By fi ltering results
through independent working groups, physicists are encouraged to criticise. Good criticism is rewarded. The goal is a reliable result, and the incentives for scientists are aligned around this goal. Weidberg worried we set the bar for results in biomedicine far too low. In particle physics, signifi cance is set at 5 sigma—a p value of 3 × 10–7 or 1 in 3·5 million (if the result is not true, this is the probability that the data would have been as extreme as they are). The conclusion of the symposium was that something must be done. Indeed, all seemed to agree that it was within our power to do that something. But
as to precisely what to do or how to do it, there were no fi rm answers.

Those who have the power to act seem to think somebody else should act first.

[Wilshire]

I think you have posted other related articles before. Its fairly terrifying to think about.  Maybe it was you that also posted about the rarity of repeatable results, that most studies are either never checked or when checked, unverified.

"Perhaps half" seems like a huge amount, and I want to say that there is no way its true, but it unfortunately doesn't seem that unlikely. "Poor methods get results" sums it up very nicely. In a society obessed with instant gratification, there are just too many drivers for bad habits.