TL-CStrans Net: a vision robot for table tennis player action recognition driven via CS-Transformer

Area of Science:

• Robotics and Artificial Intelligence in Sports
• Computer Vision and Machine Learning
• Neuroscience and Human-Computer Interaction

Background:

• Robotics technology is increasingly used in sports, but traditional methods often overlook textual data.
• Existing sports analysis relies heavily on image or video, limiting comprehensive understanding.
• Integrating multimodal data can significantly improve the accuracy and depth of sports performance analysis.

Purpose of the Study:

• To develop a multimodal vision robot for accurate table tennis player action recognition.
• To effectively integrate visual and textual information for enhanced sports training analysis.
• To demonstrate the efficacy of combining CS-Transformer, CLIP, and transfer learning for this task.

Main Methods:

• Utilized CS-Transformer as the neural computing backbone for processing visual data from table tennis scenes.
• Employed the CLIP model to jointly learn image and text representations, aligning visual and textual modalities.
• Leveraged transfer learning with pre-trained CS-Transformer and CLIP models to reduce training requirements.

Main Results:

• TL-CStrans Net demonstrated outstanding performance in table tennis stroke recognition.
• The multimodal approach successfully integrated visual and textual information for improved accuracy.
• Transfer learning significantly reduced computational and training demands.

Conclusions:

• The proposed TL-CStrans Net offers a significant advancement in sports robotics and action recognition.
• This research highlights the potential of multimodal approaches in bridging computer vision, neural computing, and neuroscience.
• The findings are important for advancing robotics technology in sports training and competition analysis.