Impact of Multi-View Fusion and Biomechanical Modeling on Markerless Motion Tracking

Markerless motion tracking using computer vision offers a scalable alternative to traditional marker-based analysis. While insight on how different emerging solutions compare could inform adoption across applications by highlighting accuracy-complexity tradeoffs, comprehensive benchmarking of these methods on the same open dataset remains a gap. This study evaluated thirteen single-view and two multi-view markerless motion capture methods against marker-based motion tracking for lower-extremity kinematics. Twenty-three healthy adults performed walking and lower-limb functional exercises, recorded with 20 infrared and 10 red-green-blue (RGB) cameras. The best-performing single-view model (WHAM) was used to test the hypothesis that the anatomical constraints imposed by biomechanical models improve the accuracy of vision-only solutions. We found that the addition of biomechanical modeling (bioWHAM) does not significantly improve overall kinematics accuracy, with root-mean-square differences (RMSD) sometimes being lower for the baseline model (WHAM) and sometimes for bioWHAM, with the median fluctuations across tasks being under 1.7$^\circ $, in either direction. Multi-view methods generally outperformed single-view ones: OpenCap, an open-source solution using two cameras, outperformed WHAM by 1.7$^\circ $ (p < 0.0001), whereas Theia3D, a commercial software using ten cameras, outperformed OpenCap by 1.3$^\circ $ (p < 0.0001). These findings suggest that while multi-view systems can enhance accuracy, the marginal benefits may not justify the added complexity across all applications. In addition to helping users consider the hardware, software, and accuracy tradeoffs, these findings highlight the need for continued innovation in multi-view fusion and incorporation of biomechanical modeling and computer vision. The accompanying dataset (I-MOVE-23) should facilitate continued benchmarking of emerging solutions.