Observational studies comparing the outcomes of different cancer therapies may be susceptible to selection biases, throwing doubt on their results, a new US review suggests.

Whereas randomised clinical trials are considered the gold standard for determining the effectiveness of new cancer treatments, observational studies involving analyses of available data to make treatment comparisons have also been used to provide information on how well patients respond to particular drugs, notes the American Cancer Society (ACS).

The review by Dr Sharon Giordano and colleagues at the University of Texas MD Anderson Cancer Center in Houston was published in the Society’s peer-reviewed journal Cancer.

Many investigators conduct observational studies by analysing data from the Surveillance, Epidemiology and End Results (SEER) Tumor Registry, a national population-based registry that collects cancer-related information, the ACS points out. Dr Giordano and her colleagues re-analysed previously published data from the SEER registry to compare the effectiveness of different cancer therapies in terms of prolonging survival.

In each case they came up with “improbable results”, the ACS said. For example, an analysis of SEER registry data from more than 5,000 patients on androgen deprivation, a hormone therapy shown in randomised clinical trials to improve survival in men with stage III prostate cancer, found that men treated with androgen deprivation actually had a higher risk of death from prostate cancer than men who did not receive the therapy.

In another re-analysis, the researchers examined data from a previously published study on the effects of fluorouracil-based chemotherapy in colon cancer. Here, Dr Giordano et al came to the same conclusion as the original study: that chemotherapy for node-positive colon cancer is associated with improved survival. But they also found that the link between treatment and survival was strongest for non-cancer deaths.

The authors attributed the improbable results seen in their analyses to selection bias in the way cancer patients are treated. For example, patients with poorer prognoses may be more likely to receive an effective drug or those with better underlying health may be given a more toxic treatment that they can tolerate.

Their findings “suggest that the results of observational studies of treatment outcomes should be viewed with caution”, the authors commented.

Analyses of observational data should at the least attempt to segregate patient outcomes into those that could possibly be due to treatments and those that could not be so, they suggested. Many observational studies on cancer treatments report only all-cause death rates and do not specify cancer-related deaths.