Recently Neuhauser and Dias raised an important question [1]: are randomised clinical trials (RCT) necessary in quality improvement?

They conclude that “RCTs need to be embedded in generalized replicatable theory. Otherwise it is a scientific house without foundation” and argue that RCTs comparing drugs cannot be replicated for two reasons: (1) after a decade or two the control arm has changed and (2) replication may be considered unethical if the original trial showed a difference. Thus it is tempting to draw the conclusion that the authors suggest that RCT should not be used when the effect of a new drug is to be tested.

Why is it so important that an experiment can be replicated? Say drug B is shown to be superior to drug A, and later drug C is shown to be superior to drug B. Then, who really cares whether the experiment comparing drug A and B can be replicated or not?

The authors’ claim that RCTs cannot be replicated may be challenged. If the conditions for the control arm has changed then the intervention arm conditions are likely to have changed in parallel over time. Then we expect to be able to replicate the findings. But we have no way of knowing this. In fact, given that the same drug is used over the years, it is unethical not to replicate the experiments regularly, according to paragraph 6 in the introduction to the Helsinki Declaration (www.wma.net/e/policy/b3.htm).

We agree with the authors that statistical process control (SPC) has an important role to play in medicine [2]. But we feel that the use of SPC for the comparison of drugs is quite risky due to the potential for the introduction of all kinds of bias [2,3]. Another reason why SPC may be inferior to RCTs is that the application of SPC requires the processes studied to be brought in statistical control prior to the intervention [2]. This may not be possible to achieve because the patient mix may vary over time. It may be argued, then, that statistical risk adjustment may take care of this problem. However, it is well known that this approach is fraud with problems [2,3]. By contrast, the attainment of statistical control is not an issue in the case of RCTs, precisely due to the random assignment of patients to the intervention groups.

References 1. Neuhauser D, Dias M. Quality improvement research: are randomized trials necessary? Qual Saf Health Care 2007; 16:77-80. 2. Winkel P, Zhang NF. Statistical Development of Quality in Medicine. John Wiley and Sons Inc, 2007; 1 – 263. 3. Deeks JJ, Dinnes J, D’Amico R, Sowdon AJ, Sakarovitch C, Song F, Petticrew M, Altman DG. Evaluating non-randomised intervention studies. Health Technol Assess 2003;7:1-173.