Clinical trials are not a reliable evidence base for the cost-effectiveness of preventive medicines such as statins in medical practice, an analysis published in the BMJ argues.

The gap between the “ideal” world of the randomised controlled trial and the messier one of actual clinical circumstances “raises the question of how well our most widely used preventive drugs work in real life”, say Teppo Järvinen, orthopaedic resident in the University of Tampere’s Department of Orthopaedic Surgery, and colleagues from Finland and Canada in their paper published on

Despite claims that important preventive drugs such as statins, antihypertensives and bisphosphonates are cost effective, there are “no valid data on [their] effectiveness, and particularly the cost-effectiveness, in usual clinical care”, Järvinen et al contend. “Despite this dearth of data, the majority of clinical guidelines and recommendations for preventive drugs rest on these claims.”

Randomised trials select patients “who are carefully diagnosed, have a carefully defined risk profile for the event being evaluated ... do not have other serious illnesses, and are likely to adhere to the treatment,” the authors comment. “Also, the study treatment is prescribed by doctors who adhere to the study protocol and participants receive special attention from dedicated staff.”

The effectiveness of treatments in clinical practice, on the other hand, is influenced by at least five factors, Järvinen et al add: the clinical population treated, diagnostic accuracy, provider compliance, patient adherence, and healthcare service coverage.

Population characteristics (e.g., age, sex) in a randomised trial, for example, “generally diverge considerably from that of the clinical population”, while in real life patients typically take fewer than half of prescribed treatments, compared with the 90% or so compliance commonly seen in efficacy trials.

von Staa assessment

The authors point to a recent assessment by von Staa and colleagues of the external validity of published cost-effectiveness studies on selective COX-2 inhibitors for inflammatory conditions.

Staa et al compared the data used in these studies (typically from randomised trials) with observed clinical data. The trial data suggested that the cost of avoiding one adverse gastrointestinal event by switching patients from conventional non-steroidal anti-inflammatory drugs to COX-2 inhibitors would be around US$20,000 (£12,500).

Yet when the same analysis was conducted using the UK’s General Practice Research Database, the cost of preventing a single bleed was fivefold greater ($104,000). Staa et al concluded that the published cost-effectiveness analyses of COX 2 inhibitors “neither had external validity nor represented the patients treated in clinical practice”, Järvinen et al observe.  

Bisphosphonate evidence

Turning to the use of bisphosphonates to prevent hip fractures in older people, the authors note claims that these drugs are as cost-effective as preventive therapies for hypercholesterolaemia or hypertension. Expert panels have come to the same conclusion based on estimates from post-hoc Markov economic models.

The fundamental problem with these models, though, is that the data showing the drug’s efficacy do not reflect clinical practice, Järvinen et al argue. “In essence, the models extend the highly specialised trial evidence on drug efficacy as if it were widely applicable in community practice.”

As such, the fracture risk-reduction data derived from specific randomised trials (e.g., 1-2% reductions in absolute risk) are “applied to a wide population largely irrespective of age, sex, co-morbidity, bone status, or previous history of fracture”

This, the authors contend, “is a far cry from reality”. The evidence that bisphosphonates prevent hip fracture is actually “very limited”, with marked  reductions seen in a restricted subpopulation of women aged 65-80 years with osteoporosis or previous fractures.

By contrast, evidence is lacking for efficacy among the people most likely to have hip fracture, the authors say – those aged _80 years and those living in nursing homes.

“And although osteoporosis is considered a predominantly female disease, about 40% of age-related fractures occur in elderly men,” Järvinen et al add. “Despite this, all current analyses of cost-effectiveness of bisphosphonates assume a universal reduction in fracture risk among all older people.”

Using data from 2003 on 7,411 hip fractures in Finland, the authors estimate that giving bisphosphonates to all 1.86 million citizens aged 50 years and over would only guarantee the prevention of 343 fractures.

Gaming the system

“We need to put an end to this kind of gaming of the system and start to advocate true comparative effectiveness research,” Järvinen et al insist.

“All relevant parties (doctors, patients, patient advocacy groups, drug industry and government regulatory bodies) should acknowledge that it is everyone’s responsibility to ensure that we have true cost-effectiveness data on all preventive healthcare before it is approved for wider use and reimbursed by the government. This responsibility should not fall on the drug industry alone.”