An epidemic of false claims

Over recent decades there have been increasing signs of concern about scientific integrity, particularly in medical and climate research, but in many other scientific areas too.

This is a large and complex issue where those interested have to do their own digging, because much of the material lies below the mainstream radar. Evidently journalists find it quicker and easier to copy and paste the official scientific press release rather than check it out.

A name of particular significance in medical science is that of Professor John P. A. Ioannidis. Last year (June 2011) Ioannidis published An Epidemic of False Claims in Scientific American.

False positives and exaggerated results in peer-reviewed scientific studies have reached epidemic proportions in recent years. The problem is rampant in economics, the social sciences and even the natural sciences, but it is particularly egregious in biomedicine. Many studies that claim some drug or treatment is beneficial have turned out not to be true. We need only look to conflicting findings about beta-carotene, vitamin E, hormone treatments, Vioxx and Avandia. Even when effects are genuine, their true magnitude is often smaller than originally claimed.

Much research is conducted for reasons other than the pursuit of truth.

Note the reference to Avandia which as the BBC reported recently (July 2^nd 2012) was one of the drugs involved in GlaxoSmithKline’s record $3 billion fine for off-label marketing.

GlaxoSmithKline is to pay $3bn (£1.9bn) in the largest healthcare fraud settlement in US history. The drug giant is to plead guilty to promoting two drugs for unapproved uses and failing to report safety data about a diabetes drug to the Food and Drug Administration (FDA). The company also conceded charges that it held back data and made unsupported safety claims over its diabetes drug Avandia.

An extract from one of the comments on the Epidemic of False Claims article is worth noting too.

The second real problem here is the almost complete breakdown in the peer review system. Papers with unfounded claims and unsubstantiated positive results are being passed routinely for publication. The bar has been lowered continuously over the last 25 years and the criteria for publication has [sic] been diluted.

In 2005 Ioannidis published a paper titled Why Most Published Research Findings Are False which has been the most downloaded technical paper from the journal PLoS Medicine.

• False findings may be the majority or even the vast majority of published research claims

• The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true

• The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true

In 2010 The Atlantic published an article on him titled Lies, Damned Lies, and Medical Science.

Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong. So why are doctors—to a striking extent—still drawing upon misinformation in their everyday practice? Dr. John Ioannidis has spent his career challenging his peers by exposing their bad science.

But it isn’t just medical science. Inadequate protocols, statistical errors, lack of reproducibility and outright fraud are gradually seeping into our perceptions of how science is done in a world with far too many second-rate scientists churned out by university PhD mills. Added to that are numerous temptations and opportunities for those with little talent, grants to earn and enough ambition to lower the bar even further.

Scattered and poorly reported the examples may be, but there is plenty of evidence out there and it seems to be growing. If there are financial and professional gains to be had from lowering standards, then those standards will be stomped into the ground by someone, somewhere.

Scientists are no longer perceived exclusively as guardians of objective truth, but also as smart promoters of their own interests in a media-driven marketplace.

Haerlin & Parr, Nature, 1999 [Greenpeace members. Well they should know.]

The scale of the problem isn’t easy to judge, because it is still not widely reported in spite of all the evidence out there. Even the corruption of climate science, by far the worst and most obvious example of modern times, is not well reported and ignored entirely by the BBC.

Yet here we have a sub-prime example of the wholesale collapse of scientific integrity. We have seen the usurpation of evidence by policy - abetted by greed, vanity and the most mind-boggling stupidity.

Finally on a slightly lighter note we have J Scott Armstrong on publication bias, or the problems encountered in trying to rock the scientific boat. Armstrong’s paper is thirty years old this year - a tongue in cheek look at the issue, but does anyone think the situation is better today than it was a generation ago?

Barriers to Scientific Contributions: The Author’s Formula:

1. Do not pick an important problem

2. Do not challenge existing beliefs

3. Do not obtain surprising results

4. Do not use simple methods

5. Do not provide full disclosure

6. Do not write clearly