Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Sim-to-Real Domain Adaptation for Early Alzheimer's Detection from Handwriting Kinematics Using Hybrid Deep Learning.

Sensors (Basel, Switzerland)·2026

Same author

Enhancing InceptionResNet to Diagnose COVID-19 from Medical Images.

Current medicinal chemistry·2025

Same author

Computer model of a spectrum analyzer for a virtual laboratory: development and introduction to the educational process.

PeerJ. Computer science·2022

Same author

Detection of COVID-19 using deep learning on x-ray lung images.

PeerJ. Computer science·2022

Same author

Data science for analyzing and improving educational processes.

Journal of computing in higher education·2021

Same author

The adoption of an e-learning system using information systems success model: a case study of Jazan University.

PeerJ. Computer science·2021

Search research articles

Related Experiment Video

Updated: Jun 21, 2025

Creating Virtual-hand and Virtual-face Illusions to Investigate Self-representation

Creating Virtual-hand and Virtual-face Illusions to Investigate Self-representation

Published on: March 1, 2017

The real-time hand and object recognition for virtual interaction.

Madi Nuralin¹, Yevgeniya Daineko², Shadi Aljawarneh³

¹Mixed Reality Laboratory, International Information Technology University, Almaty, Kazakhstan.

Peerj. Computer Science

|July 10, 2024

Summary

This study introduces a real-time system for recognizing hand-object interactions in extended reality (XR) using a single RGB camera. The approach enables natural virtual interactions with objects like steering wheels.

Keywords:

Computer vision EXtended reality (XR)Hand-object interactions Mediapipe Virtual interactions

More Related Videos

Author Spotlight: Insights into the Analysis of Human Interaction with 3D Virtual Objects

Author Spotlight: Insights into the Analysis of Human Interaction with 3D Virtual Objects

Published on: October 18, 2024

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping

Published on: April 21, 2023

Related Experiment Videos

Last Updated: Jun 21, 2025

Creating Virtual-hand and Virtual-face Illusions to Investigate Self-representation

Creating Virtual-hand and Virtual-face Illusions to Investigate Self-representation

Published on: March 1, 2017

Author Spotlight: Insights into the Analysis of Human Interaction with 3D Virtual Objects

Author Spotlight: Insights into the Analysis of Human Interaction with 3D Virtual Objects

Published on: October 18, 2024

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping

Published on: April 21, 2023

Area of Science:

Computer Vision
Human-Computer Interaction
Extended Reality (XR)

Background:

Recognizing hand-object interactions is challenging due to variability and occlusion.
Estimating 3D hand positions from single frames is difficult, especially with obstructed views.

Purpose of the Study:

To develop a novel, real-time approach for recognizing hand-object interactions in XR environments.
To facilitate virtual interactions with physical objects, using a steering wheel as a case study.

Main Methods:

A pipeline combining MediaPipe for hand landmark detection.
Faster Region-based Convolutional Neural Network (Faster R-CNN) for steering wheel tracking.
A gesture recognition module to analyze hand-steering wheel interactions using single RGB camera data.

Main Results:

Demonstrated natural and realistic interaction between physical objects and virtual environments.
Showcased precision and stability in recognizing hand-object interactions in a steering wheel manipulation scenario.
Validated the real-time performance of the proposed model pipeline.

Conclusions:

The developed system effectively recognizes hand-object interactions in XR.
This approach enhances user experience in virtual environments and has potential applications in reducing real-world emissions.
The system provides a stable and precise method for virtual object manipulation.