Generalisability of clinical trials in otitis media with effusion

The impact of a randomised controlled trial (RCT) upon practice depends on its external validity (generalisability). This paper summarises and illustrates a framework for judging and augmenting external validity, emphasising its application to treatment trials in otitis media with effusion (OME) so as to permit stronger inferences in the future.

The external validity of two surgical trials in the field of OME (TARGET, UK and KNOOP-3, the Netherlands) has been examined within a framework emphasising effect modification, in four specific ways: (1) comparison of the demographic characteristics of the trial population with the domain population; (2) studying the distributions on possible effect modifiers (i.e. variables conditioning the benefit from intervention); (3) studying whether effect modification occurs in the analyses; and (4) comparing outcome measures between the randomised and the eligible but non-randomised children.

For neither KNOOP-3 and TARGET were large discrepancies found between randomised and non-randomised children for any of the demographic variables. Differences in distributions along possible effect modifiers were found, but the overlaps were large enough for it still to be possible to study whether these factors indeed modified the outcome. Results for the randomised and non-randomised but eligible patients were similar. The results of both trials therefore appear to be generalisable to their domain populations.

A superficial contrast in the results (KNOOP null, TARGET positive) does not amount to a contradiction, because of differences in the clinical question appropriate to the respective age and populations defined. Attention to quality of design and external validity of randomised controlled trials should achieve higher applicability.