Mixed reality simulation of guided needle insertion: effects of interaction and visual fidelity

Mixed reality (MR) simulations are increasingly used in medical training to provide safe, controllable, and repeatable practice environments. While visual fidelity (VF) and interaction fidelity (IF) are known to influence user experience (UX), their respective effects on performance and UX in complex, precision-based medical procedures remain insufficiently understood.

We conducted a within-subjects user study investigating the effects of VF (high vs. low) and IF (tangible vs. virtual interaction) in a precise needle insertion task. A non-immersive setup served as a reference condition. Performance metrics, perceived realism, workload, and UX were assessed across conditions.

Both fidelity factors successfully modulated perceived realism. Tangible interaction significantly improved insertion depth control, reduced workload, and enhanced UX. High VF primarily increased spatial presence and involvement, with only limited benefits for task performance. Interaction effects revealed that tangible interaction provided greater advantages under low VF, particularly in terms of task completion time and needle alignment.

Visual and interaction fidelity play distinct and complementary roles in MR-based medical training. Tangible interaction predominantly supports procedural accuracy and efficiency, whereas high visual fidelity mainly enhances immersive engagement. MR system design should therefore align fidelity decisions with specific training objectives.