Pharmaceutical companies accustomed to allegations of bias in industry-funded clinical trials can take some comfort from a new study published in the BMJ.
A US team led by Veronica Yank, clinical instructor at California’s Stanford University, conducted a retrospective cohort study of meta-analyses on antihypertensive trials to determine whether financial ties to a single drug company were associated with favourable results or conclusions.
Previous studies have shown that both randomised controlled trials and, latterly, meta-analyses by researchers with financial ties to drug companies are more likely to produce results and conclusions favouring the sponsor’s products, Yank et al noted in the BMJ (Online First publication).
In the case of antihypertensives, though, they found that meta-analyses by researchers with financial ties to a single drug company were not associated with favourable results. They were, however, significantly more likely to come to favourable conclusions, even when differences in study quality were taken into account.
Meta-analyses are “the highest level of research evidence in the hierarchy of study types” and they may equal, if not surpass, randomised controlled trials in both their cost-effectiveness and their influence on patient care and healthcare policy, Yank et al comment. Moreover, drug companies have started to reference meta-analyses in their advertisements, the authors point out.
Poor concordance
They looked at 124 meta-analyses on antihypertensive drugs published up to December 2004, 49 (40%) of which involved financial ties to a single drug company. Among the meta-analyses with financial ties to one drug company, 27 (55%) generated favourable results and 45 (92%) favourable conclusions – in other words, in 18 (37%) of these cases there was poor concordance between the results and the conclusions.
For all other types of meta-analyses, including those with financial ties to multiple drug companies, the discrepancy between results and conclusions was 8% or six out of a total of 75 studies. Quite a high proportion – 49 or 65% – of these meta-analyses produced favourable results but the conclusions were more closely matched, coming out in favour of the drug in 55 or 73% of cases.
The rate of favourable results for studies with financial links to more than one drug company was 57% (eight out of 14) and the rate of favourable conclusions was 79% (11 studies), giving a more muted 21% score for poor concordance between results and conclusions.
No discrepancy
Where the meta-analysis was conducted by a non-profit institution, there was no discrepancy at all between favourable results and conclusions – both occurred in 78% or 21 of a total of 27 studies. Even when the meta-analysis was conducted by a non-profit institution in conjunction with a drug company (nine studies), the scores came out the same: six or 67% of the meta-analyses produced favourable results and 67% generated favourable conclusions.
While accepting the limitations of confining their research to just one clinical topic, the researchers argued that their findings had “considerable relevance to the real world … as the marketing of antihypertensive drugs constitutes a multibillion dollar-a-year industry, and antihypertensives are some of the most prescribed drug classes in the world”.
Not only did the poor concordance between results and conclusions in some of the meta-analyses suggest these study types were, like others, open to “the influence of systematic bias”, the authors commented, but they also exposed “a failure of peer review”. Both editors and peer reviewers must have read manuscript versions of the analyses presenting discordant results and conclusions, yet “they did not prevent publication of biased conclusions”, Yank et al pointed out.
Editors and peer reviewers, as well as policymakers, meta-analysts and readers, “should closely scrutinise the conclusions of meta-analyses to ensure that they are supported by the data”, they warned.
More is more
In an accompanying BMJ editorial, Richard Epstein, professor of law at the University of Chicago Law School in Illinois, said the observed biases raise the question of whether it was better to restrict the role of drug companies in financing and organising clinical trials, resulting in “fewer studies of presumably better quality”, or to stick with the status quo and have “more studies whose quality may be more biased”.
Professor Epstein, who has worked for many years as a consultant to drug companies, plumped decisively for the latter option. “Nothing in the work of Yank and colleagues suggests that the raw data from the drug-sponsored studies were defective,” he commented. “The criticisms are directed to the optimistic inferences drawn from data. But these inferences are drawn from publicly available sources, which other investigators could presumably check without having to re-collect the original data from scratch.”
A sensible approach, he suggested, might be “to encourage further dialogue by asking for editorial comment” – which did not have to appear in the same journals as the original studies. As long as the point of contention was the interpretation and not the collection of clinical trial data, “the solution is not state regulation”, Professor Epstein argued.
Rather, he proposed, doctors should be warned about exercising caution in their interpretation of the studies’ conclusions. “The medical profession already has voluntary means to improve its performance,” Professor Epstein said.
