Related Experiment Videos
The Linguistic Analysis of Scene Semantics: LASS
1Psychology Department, Northeastern University, Boston, MA, USA. rose.dy@husky.neu.edu.
Behavior Research Methods
|April 30, 2020
Summary
We introduce Linguistic Analysis of Scene Semantics (LASS), a novel computational linguistics method to analyze object-scene relationships. LASS accurately maps semantic similarity in images, proving useful for scene understanding and novelty detection.
Area of Science:
- Computer Vision
- Computational Linguistics
- Artificial Intelligence
Background:
- Analyzing object-scene contextual relationships is crucial for image understanding.
- Existing methods may lack flexibility or automation in semantic analysis.
Purpose of the Study:
- To introduce Linguistic Analysis of Scene Semantics (LASS), a new computational linguistics method.
- To enable automated and flexible analysis of object-scene semantic relationships in images.
Main Methods:
- Utilized Facebook Research's fastText language model for linguistic semantic similarity.
- Developed semantic similarity maps by embedding scores into object segmentation masks.
- Enabled full automation using deep learning for label and mask generation.
- Compared human and neural network annotations on the LabelMe database.
Main Results:
- LASS generates semantic similarity maps with desirable properties.
- Automated LASS maintains high spatial and semantic similarity to human annotations.
- Evaluated semantic consistency, revealing uniform distributions of relatedness across image regions.
Conclusions:
- LASS is an accurate, automatic, flexible, and useful method for analyzing object-scene semantics.
- Findings suggest contextually appropriate objects are uniformly distributed in images.
- LASS has potential applications in scene grammar and novelty detection research.