Hippocampal Memory-Like Separation-Completion Collaborative Network for Unbiased Scene Graph Generation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Scene Graph Generation (SGG) is a complex cross-modal task requiring simultaneous entity and relationship identification.
• The long-tailed distribution of real-world data causes SGG models to be biased towards common relations, neglecting rarer ones.
• Existing methods focus on data re-balancing or feature refinement, but fail to address the core issue of catastrophic interference.

Purpose of the Study:

• To propose a novel modeling-level solution for the long-tailed problem in Scene Graph Generation.
• To introduce a network architecture inspired by hippocampal memory processes to combat catastrophic interference.
• To enhance the accurate generation of scene graphs by improving the representation of rare relationships.

Main Methods:

• Developed the Hippocampal Memory-Like Separation-Completion Collaborative Network (HMSC2) to mimic hippocampal encoding and retrieval.
• Implemented Gradient Separation Classifier and Prototype Separation Learning to model separated classifiers and prototypes, reducing interference for tail categories.
• Introduced a Prototype Completion Module and Contrastive Connected Module to supplement incomplete information and connect representations in hypersphere space.

Main Results:

• HMSC2 achieved state-of-the-art performance on the Visual Genome and GQA datasets.
• The proposed method effectively alleviated the long-tailed problem in Scene Graph Generation.
• Demonstrated significant improvements in unbiased SGG task performance.

Conclusions:

• The HMSC2 network offers a novel and effective approach to address the long-tailed issue in Scene Graph Generation.
• The memory-inspired mechanism provides a promising direction for future research in tackling data imbalance in AI tasks.
• The successful application on benchmark datasets validates the efficacy of the proposed method.