The Department of Health has promised changes to the GP Survey that enraged family doctors when it emerged in June.

Pulse reports that the DH is to rework the survey, which had the effect of penalising some GPs on the basis of a low response rate. But it is evident from a paper published in BMC Family Practice (an open-access journal) that the problems ought to have been anticipated. The paper, by a team led by Professor Martin Roland of the University of Cambridge, describes the development of the survey and the results of piloting it. Among the co-authors are Professor John Campbell of Peninsula Medical School in Exeter, and Patten Smith and Sonja Nissen of Ipsos-Mori, the company that carried out the survey for the DH. 

BMC Family Practice publishes the responses of the referees who read the paper before publication. Both warn of low response rates skewing results, and ask for this to be made more explicit. Stewart Mercer says the results of many items are “massively skewed” and the poor response rate “glossed over”. No attempt had been made to compare responders (just under 40 per cent) with the 60 per cent who did not respond. “How do we know that the 40 per cent who reply are not highly unrepresentative of the population?” he asks.
 
Chris J Salisbury, the other referee, says that the response rate of 39.4 per cent was effectively 34 per cent after taking account of missing data for each question. “The authors point out that this is not unusual for postal questionnaires” he says. “That may be true, but there is no discussion in the paper of the possible reasons for this or the implications ...the key issue is non-response bias, ie whether the minority of people who responded to the questionnaire are representative of all the patients of a practice.”
 
Pulse says its analysis showed that practices lost about £35 million in annual funding as a result of the survey and other changes made at the same time, and some are appealing to primary care trusts for reimbursement.
 
One odd aspect of the pilot study is that half the patients were asked questions about their religion and sexuality, and half were not, to see if asking such questions lowered the response rate. It did, but not significantly, and the DH intends to ask these questions in future, “with a view to informing government policy regarding inequalities in relation to patient experience of care”. It’ll be up to patients to decide if they want to answer them. Some, no doubt, will consider such question intrusive or even improper, from a Government department
.