The acres of media coverage accorded last month to positive results from a late-stage trial of a combination AIDS vaccine in Thailand were only part of the story, it has emerged.

The Wall Street Journal and Science magazine have put a spanner in the works by suggesting that a secondary analysis of data from the trial, which was sponsored by the US Army Surgeon General, dilutes the already modest efficacy data announced at a press conference on 24 September and takes them below the level of statistical significance.

Moreover, it is claimed, researchers knew at the time of the press conference about the less impressive results from the per protocol analysis of the study. AIDS Healthcare Foundation, the largest non-profit provider of HIV/AIDS healthcare in the US, said the “premature and partial reporting of select – and favourable – vaccine trial data here underscores an inherent and glaring conflict of interest”.

The six-year trial on more than 16,000 adult volunteers involved Sanofi Pasteur's ALVAC HIV vaccine and AIDSVAX, developed by VaxGen and licensed to Global Solutions for Infectious Diseases. The results caused such a stir because they were the first to show that a vaccine could be effective at preventing HIV infection.

The prime boost combination of the two treatments lowered the rate of HIV infection by 31.2% compared with placebo. As The Wall Street Journal (WSJ) noted, this was based on a ‘modified intent to treat’ analysis, which included most of the trial enrollees regardless of whether they complied with the trial regimen.

The number of infected participants was small – 51 in the vaccine arm compared with 74 in the placebo group – but the probability of the outcome being down to chance was 3.9% and the results achieved statistical significance.

In the more stringent and hitherto unpublicised ‘per protocol’ analysis, which focused on those participants given the full complement of vaccine shots in accordance with the trial design, a total of 86 people in either the vaccine or the placebo group were infected with HV. The efficacy rate was 26.2% and the probability of the result being due to chance was 16% (i.e., P value = 0.16), the WSJ reported.

This took the results well out the range of statistical significance. The newspaper cited two unidentified “AIDS scientists” who had seen the per protocol analysis.

Full results from the combination vaccine trial will be presented at the AIDS Vaccine 2009 conference in Paris on 20 October. In the meantime, the revelations from Science magazine and the WSJ have raised questions about why the trial sponsor and supporters, which included the US National Institute of Allergy and Infectious Diseases (NIAID, part of the US National Institutes of Health), the Thai Ministry of Public Health and Sanofi Pasteur, jumped the gun and gave an incomplete picture of the study outcome.

Dr Anthony Fauci, director of the NIAID, was quoted in The New York Times as saying that putting multiple biostatistical analyses in a news release "would have confused everybody", while insisting the intent-to-treat analysis was "the gold standard".

There are concerns that the apparent recourse to ‘spin’ will backfire and rob a still promising trial of any credibility, jeopardising support for AIDS vaccine research.
“What is needed is rigorous review and evaluation of such vaccine trial data by an independent outside body,” commented Michael Weinstein, president of AIDS Healthcare Foundation.

“For NIH-funded scientists or US government researchers to also evaluate and discuss the significance of their own research is akin to allowing students to grade their own papers,” he added. “Data can be cherry-picked; more so, if there is an incentive for future or increased government research funding based on today’s thought-to-be ‘promising’ outcomes.”