A Universal Decoupled Training Framework for Human Parsing | JoVE Visualize

Area of Science:

Computer Vision
Robotics
Machine Learning

Background:

Human parsing is crucial for human-robot interaction, but current datasets exhibit imbalanced, long-tailed distributions.
Category similarity in datasets leads to inaccurate human parsing predictions, hindering system performance.
Existing methods struggle to effectively address the challenges posed by imbalanced data in human parsing.

Purpose of the Study:

To propose a novel decoupled training framework, Decoupled Training framework based on Pixel Resampling (DTPR), to overcome the long-tailed distribution issue in human parsing datasets.
To introduce a new sampling method, Pixel Resampling based on Accuracy (PRA), specifically designed for semantic segmentation within the DTPR framework.
To enhance the feature extraction capabilities and improve performance on tail categories in human parsing models.

Main Methods:

A two-phase decoupled training framework (DTPR) was developed, separating feature extraction enhancement from tail category performance improvement.
A novel Pixel Resampling based on Accuracy (PRA) sampling strategy was integrated into the DTPR framework for semantic segmentation.
The proposed methods were evaluated on MHPv2.0 and LIP datasets, testing across high-precision and real-time state-of-the-art (SOTA) models.

Main Results:

The DTPR framework significantly improved the Mean Average Precision (MPA) metric by over 6% on both MHPv2.0 and LIP datasets.
The Mean Intersection over Union (mIoU) metric saw an increase of more than 1% without any modifications to the underlying model architecture.
The framework demonstrated effectiveness in enhancing model performance, particularly for tail categories, without compromising overall structure.

Conclusions:

The proposed DTPR framework effectively addresses the long-tailed distribution problem in human parsing datasets.
The PRA sampling method, when combined with DTPR, enhances semantic segmentation performance, especially for minority classes.
This approach offers a robust solution for improving human parsing accuracy in real-world human-robot interaction systems.