Representing and retrieving video shots in human-centric brain imaging space

Area of Science:

• Computer Vision
• Neuroscience
• Image Processing

Background:

• Traditional video retrieval relies on low-level visual features (color, shape, texture, motion), which suffer from the semantic gap.
• This limitation hinders accurate video representation and retrieval in large-scale databases.
• Bridging the semantic gap is crucial for advancing video analysis and understanding.

Purpose of the Study:

• To develop a novel methodology for video shot representation and retrieval using human-centric, high-level features.
• To leverage brain imaging space (BIS) for capturing semantic meaning in video content.
• To improve the accuracy and effectiveness of video retrieval systems.

Main Methods:

• Utilized the dense individualized and common connectivity-based cortical landmarks (DICCCOL) system to identify functional brain networks and ROIs.
• Extracted functional connectivities between ROIs as BIS features to characterize video semantics.
• Applied feature selection, Gaussian process regression (GPR) for mapping visual to BIS features, and manifold ranking for similarity measurement.

Main Results:

• Developed a robust method for video representation and retrieval using brain-derived semantic features.
• Demonstrated superior performance compared to traditional methods on the TRECVID 2005 dataset.
• Successfully mapped low-level visual features to high-level semantic features in the BIS.

Conclusions:

• Video representation and retrieval can be significantly enhanced by incorporating human-centric features from brain imaging space.
• The proposed BIS feature approach effectively bridges the semantic gap in video analysis.
• This methodology offers a promising direction for future research in intelligent video retrieval systems.