Human Interaction Understanding With Consistency-Aware Learning.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Human activity classification has advanced significantly, but human interaction understanding (HIU) lags due to challenges in modeling complex relationships.
• Existing methods often rely on shallow graphical representations, which are insufficient for capturing intricate human interactive relations.

Purpose of the Study:

• To propose a novel deep consistency-aware framework to overcome grouping and labeling inconsistencies in human interaction understanding (HIU).
• To enhance the modeling of complex human interactive relations for improved performance in HIU tasks.

Main Methods:

• A framework integrating a backbone Convolutional Neural Network (CNN) for feature extraction.
• A factor graph network for implicit learning of higher-order consistencies among labeling and grouping variables.
• A consistency-aware reasoning module, embedding reasoning bias into energy or loss functions, trained end-to-end with an efficient mean-field inference algorithm.

Main Results:

• The proposed consistency-learning modules significantly improve HIU performance by complementing each other.
• The framework achieves leading results on three human interaction understanding benchmarks.
• Effectiveness is further validated through experiments on human-object interaction detection.

Conclusions:

• The deep consistency-aware framework effectively addresses inconsistencies in HIU, outperforming previous methods.
• The integration of implicit and explicit consistency learning is crucial for advancing human interaction understanding.
• The proposed approach offers a robust solution for complex HIU tasks and related applications like human-object interaction detection.