When authors of medical papers have no statistically significant results to report, what do they do? Resort to spin.

At the International Congress of Peer Review and Biomedical Publication in Vancouver earlier this month, Isabelle Boutron from the Centre for Statistics in Medicine at Oxford and colleagues reported a study into randomised controlled trials in which the primary outcome failed to reach statistical significance. They found 72 such trials, of which more than 40 per cent were reported in such a way as to imply a positive result.

The authors used sentences such as: “[the treatment] is expected to be a very important modality in the treatment strategy” and “[the treatment effect] approached but did not achieve conventional statistical significance”, this week's BMJ reports.
 
In another paper presented at the conference, Lisa Bero and Yolanda Chang of the University of California at San Francisco examined the language used in drug trials. They found that half the statements claiming a positive effect did not mention statistical significance, and of the other half that did, almost a third were unsupported by a statistical test. They conclude: “Rhetoric to frame research results in drug studies overstates the effectiveness of a drug.”
 
A third study suggests that journals that demand an independent statistical analysis of papers publish fewer industry-funded studies. Elizabeth Wager of the Committee on Publication Ethics and colleagues did a “before-and-after” study of a policy change at the Journal of the American Medical Association, which introduced the compulsory statistical test in July 2005.
 
They found that in the three years after the change, the percentage of RCTs in JAMA funded by industry fell from 60.1 per cent to 47.3 per cent compared to the three years before. The proportion in New England Journal of Medicine and Lancet did not show a similar decline – indeed showed a small increase. The study could not determine whether statistical checking discouraged the submission of papers, or increased the rate at which they were rejected.