[Lack of correlation between performances in a simulator and in reality]

Simulation-based training provides obvious benefits for patients and doctors in education. Frequently, virtual reality simulators are expensive and evidence for their efficacy is poor, particularly as a result of studies with poor methodology and few test participants. In medical simulated training- and evaluation programmes it is always a question of transfer to the real clinical world. To illustrate this problem a study comparing the test performance of persons on a bowling simulator with their performance in a real bowling alley was conducted.

Twenty-five test subjects played two rounds of bowling on a Nintendo Wii and 25 days later on a real bowling alley. Correlations of the scores in the first and second round (test-retest-reliability) and of the scores on the simulator and in reality (criterion validation) were studied and there was tested for any difference between female and male performance.

The intraclass correlation coefficient equalled 0.76, i.e. the simulator fairly accurately measured participant performance. In contrast to this there was absolutely no correlation between participants' real bowling abilities and their scores on the simulator (Pearson's r = 0.06). There was no significant difference between female and male abilities.

Simulation-based testing and training must be based on evidence. More studies are needed to include an adequate number of subjects. Bowling competence should not be based on Nintendo Wii measurements. Simulated training- and evaluation programmes should be validated before introduction, to ensure consistency with the real world.