Using ultrasound images of the forearm to predict finger positions

Medical ultrasound imaging is a well-known technique to gather live views of the interior of the human body. It is totally safe, it provides high spatial and temporal resolution, and it is nowadays available at any hospital. This suggests that it could be used as a human-computer interface. In this paper, we use ultrasound images of the human forearm to predict the finger positions, including thumb adduction and thumb rotation. Our experimental results show that there is a clear linear relationship between the features we extract from the images, and finger positions, expressed as angles at the metacarpo-phalangeal joints. The method is uniformly valid for all subjects considered. The unavoidable movements of the ultrasound probe with respect to the skin and of the skin with respect to the inner musculoskeletal structure are compensated for using the optical flow. Typical applications of this system range from teleoperated fine manipulation to finger stiffness estimation to ergonomy. If successfully applied to transradial amputees, it could be also used to reconstruct the imaginary limb, paving the way to, e.g., fine control of hand prostheses, treatment of neuropathic/phantom limb pain and visualization of the imaginary limb as a tool for the neuroscientist.