Creating and Exploring Semantic Annotation for Behaviour Analysis.

Area of Science:

• Computer Science
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Ground truth is crucial for supervised learning, knowledge-based methods, and performance evaluation in activity recognition and behavior analysis.
• Semantic annotation, which assigns meaning to labels, enhances simple symbolic labeling for advanced reasoning.
• Existing datasets like CMU's grand challenge dataset are underutilized due to incomplete ground truth annotations.

Purpose of the Study:

• To present a novel approach for semantic annotation using plan operators.
• To manually create ground truth annotations for the CMU grand challenge dataset.
• To demonstrate the utility of semantic annotation for deriving hidden properties and behavioral insights.

Main Methods:

• Developed a novel semantic annotation methodology based on plan operators.
• Manually created a step-by-step workflow for generating ground truth annotations.
• Applied the method to semantically annotate the Carnegie Mellon University (CMU) grand challenge dataset.

Main Results:

• Successfully derived hidden properties, behavioral routines, and changes in initial/goal conditions from the annotated dataset.
• Achieved high inter-rater reliability (Cohen's κ = 0.8), indicating excellent agreement between annotators.
• The annotated dataset and semantic models are publicly released to facilitate further research.

Conclusions:

• The proposed plan operator-based semantic annotation method effectively enhances ground truth data for activity recognition.
• Semantic annotation unlocks deeper analysis of behavioral datasets, overcoming limitations of incomplete existing annotations.
• Publicly releasing the annotated CMU dataset and models promotes wider use and advancement in behavior analysis research.