DeepAction: a MATLAB toolbox for automated classification of animal behavior in video

Area of Science:

• Ethology and behavioral neuroscience
• Computer vision and machine learning
• Bioinformatics and computational biology

Background:

• Automated animal behavior identification in videos is crucial for research but is labor-intensive.
• Existing methods for behavior annotation often lack accuracy or require extensive training data.

Purpose of the Study:

• To introduce DeepAction, a deep learning-based toolbox for automated animal behavior annotation in videos.
• To demonstrate the high accuracy and efficiency of DeepAction compared to human annotation and existing methods.
• To provide a reliable confidence score for automated annotations to aid human review.

Main Methods:

• Utilized a deep learning approach combining a pretrained convolutional neural network (CNN) for feature extraction and a recurrent neural network (RNN) for classification.
• Trained and evaluated the classifier on diverse datasets, including rodent and octopus behavioral videos.
• Developed a confidence score to estimate the accuracy of the automated annotations.

Main Results:

• DeepAction achieved high accuracy in identifying animal behaviors across multiple datasets.
• The system required minimal training data and outperformed human agreement and comparable existing methods.
• The confidence score accurately reflected classifier performance and reduced the time needed for human correction.

Conclusions:

• DeepAction offers a powerful, accurate, and efficient solution for automated animal behavior annotation in research.
• The open-source toolbox and annotation interface facilitate broader adoption and advancement in behavioral studies.
• This deep learning approach significantly streamlines the analysis of animal behavior videos, accelerating scientific discovery.