US study provides further evidence of publication bias

A new US study has added to the mounting pile of evidence that publication bias is routine practice in clinical research.

Researchers Kirby Lee, Peter Bacchetti and Ida Sim from the University of California San Francisco conducted a cohort study of clinical trials supporting new drugs that were approved by the US Food and Drug Administration (FDA) between 1998 and 2000.

Publication status and time from approval to full publication in the medical literature at two and five years were determined through searches of PubMed and other databases up to 1 August 2006. The results were published in the open-access journal PLoS Medicine.

Previous research has documented the problem of publication bias and incomplete or selective reporting of clinical trials submitted to licensing authorities in Sweden, Finland and the US, the authors noted. However, these analyses were confined to a single drug class, namely antidepressants.

The researchers identified a total of 909 trials supporting 90 approved drugs in the FDA reviews, of which 394 trials (43%) were published in the medical literature after a minimum of 5.5 years from FDA approval. Among those 340 trials categorised as “pivotal” – i.e., which demonstrated the efficacy and safety of a drug for its proposed indication(s) and provided the most useful information for clinical decision-making – the publication rate was found to be 76% (257).

Where the results of these trials were statistically significant – 432 or 75% of the studies analysed by Lee et al – 285 or 66% of this subset found their way into the medical literature. By contrast, the publication rate for the 144 (25%) trials defined as not statistically significant was just 52 or 36%.

Since published trial reports are often incomplete and have been shown to report favourable outcomes selectively, the published evidence supporting FDA-approved drugs “may be even more skewed than our results suggest”, the authors comment. However, provisions in the FDA Amendments Act of 2007 obliging sponsors to make public all predeclared primary and secondary trial outcomes regardless of whether they are positive or negative “should go a long way toward correcting this skew”, they believe.

While the Act should also speed up the dissemination of trial results – in the cohort study, 40% of the trials that were eventually published appeared more than one year post-approval – it could also, paradoxically, aggravate rather than reduce publication bias, the researchers warn.

With the basic results already in the public domain, sponsors may feel less compelled to undertake full publication of equivocal trials, while the time pressure under the Act to submit manuscripts within one year of approval could “focus sponsor efforts even more” on submitting positive trials and trials of most interest to journals, they suggest.