The most striking revelation in a quarter of comments in BMJ.com on the Department of Health’s risk sharing scheme for multiple sclerosis (MS) drugs is its undermining of subsequent MS appraisals by NICE. How did this come about?

First, the risk sharing scheme set the cost-effectiveness bar for MS drugs high, at £36,000 per quality-adjusted life-year (QALY). Current austerity may call into question the affordability of QALYs greater than £15,000, far less £20,000-30,000 or higher.

Secondly, whereas NICE had determined that the MS drugs in question were not cost-effective, Rafferty claims that later MS drugs have their cost-effectiveness gauged against these DH-afforded drugs, themselves not cost-effective - NICE’s appraisal having since been corroborated a fortiori by the results of the risk-sharing scheme itself. The set-up that confronts NICE is undermining of proper appraisal of cost-effectiveness if ‘Aunt Sally’ cost-ineffective comparators are imposed as the baseline.

The issue goes wider than MS drugs. UK has, in effect, shifted its NHS standard for comparison from one based on ‘efficacy’ to a control treatment which is selected on ‘efficacy and cost-effectiveness’. This is sensible, because there is no point in comparisons being made against an efficacious treatment that is unaffordable because it breaches cost-effectiveness thresholds.

A consequence is that randomized controlled trials for regulatory purposes may have to adopt different control treatments internationally according to the nationally-recognised standard treatment which may differ because cost-effectiveness thresholds differ.

Pragmatic randomized controlled trials can readily be designed to compare cost-effectiveness but their ethical basis becomes economic equipoise rather than uncertainty about efficacy. See, for example, the paper by McCabe et al. in BMJ.com and the recommendations on policy trials and cost-effectiveness considerations by Royal Statistical Society’s Working Party on Performance Monitoring in the Public Services (Recommendation 4 on cost-effectiveness, and Recommendation 9 on policy evaluation).

Disappointingly, the DH risk sharing scheme did not collect information on cognition, although MS patients had reckoned that their cognition (in a given disability state) was better on-drug than off, for which some evidence was adduced to NICE. Moreover, the monitoring of disability (EDSS scores) was itself sub-optimal by research standards despite considerable investment in staff.

Conflicts of interest: SMB served on NICE Appraisal Committee which advised that the MS drugs in DH’s risk sharing scheme were not cost-effective; and chaired Royal Statistical Society’s Working Party on Performance Monitoring in the Public Services.