Mining Interpretable AOG Representations From Convolutional Networks via Active Question Answering.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Convolutional Neural Networks (CNNs) are powerful tools for image analysis.
• Understanding the internal representations learned by CNNs, particularly object-part patterns, remains a challenge.
• Existing methods often require extensive annotated data for feature interpretation.

Purpose of the Study:

• To develop a method for mining interpretable object-part patterns from pre-trained CNNs.
• To organize these patterns using an And-Or Graph (AOG) with a semantic hierarchy.
• To create an efficient learning process that minimizes the need for manual annotations.

Main Methods:

• Mining object-part patterns from CNN conv-layers using an And-Or Graph (AOG) representation.
• Employing a question-answering (QA) method with active human-computer interaction for incremental pattern discovery.
• Utilizing the AOG for part localization and actively seeking annotations for unexplained object features.
• Gradually refining the AOG to semanticize CNN representations.

Main Results:

• The AOG successfully organizes object-part patterns into a four-layer semantic hierarchy.
• The QA method effectively mines patterns from CNNs with very few annotations (3-20).
• Experimental results show high learning efficiency, using only 1/6-1/3 of annotations compared to traditional methods.
• Achieved similar or superior part-localization performance compared to fast-RCNN.

Conclusions:

• The proposed method provides an interpretable AOG representation of CNN features.
• Active human-computer communication enables efficient and low-annotation learning of object-part patterns.
• This approach significantly enhances the semantic understanding of CNN representations.