Viewpoint-Controllable Telepresence: A Robotic-Arm-Based Mixed-Reality Telecollaboration System.

Area of Science:

• Human-Computer Interaction
• Virtual Reality
• Robotics

Background:

• Mixed-reality (MR) telecollaboration faces challenges in remote viewpoint manipulation.
• Current systems limit natural and active user control over their perspective.

Purpose of the Study:

• To develop an MR telepresence system enabling active and flexible viewpoint control for remote users.
• To enhance the perception of the local environment beyond the limitations of camera field-of-view and robotic arm range.

Main Methods:

• A robotic arm with a stereo camera allows remote users to manipulate viewpoints via head movement.
• A 3D reconstruction method combined with field-of-view enhancement expands environmental perception.
• A prototype MR telecollaboration system was built and evaluated through two user studies.

Main Results:

• User Study A demonstrated improved interaction efficiency and user experience compared to 360 video and first-person view techniques.
• The system effectively enhanced remote users' ability to actively observe the local environment.
• User Study B provided comprehensive feedback on the MR telecollaboration system from both local and remote perspectives.

Conclusions:

• The proposed telepresence system significantly improves interaction efficiency and user experience in MR telecollaboration.
• The integration of robotic viewpoint control and 3D reconstruction offers a more immersive and effective telepresence solution.
• Findings provide valuable insights for future development of advanced MR telecollaboration systems.