Real-time rodent behavior classifier using color-based body segmentation (R2C2)

The heavy reliance of animal studies on human observers makes them prone to observer-specific biases. To mitigate such shortcomings, much research has been focused on computerizing the characterization of animal behavior. Such automation can lead to more reliable and cost-effective behavior quantifications. Yet, there remain challenges in developing end-to-end solutions that allow users to easily train custom behavioral classifiers with minimal data while maintaining low computational demands. Here we resolve these challenges through a rodent behavior classifier, the real-time rodent behavior classifier using color-based body segmentation (R2C2) algorithm, which uses color-based body segmentation to track rodent body parts and consequently their behaviors. Based on the 'hue, saturation, value' (HSV) color difference in furs or exposed skins, the R2C2 creates simple white-black color boundaries for each body part, which are then used to discern and track body parts in real time to extract movement-based features. We combined wavelet transform-based tracking with HSV color-based body part segmentation to substantially reduce computational requirements while minimizing the number of input features needed for classification. Loading these features into our convolutional neural network algorithm, the R2C2 achieves performance on par with an expert human observer. Furthermore, it can differentiate subtle behavioral patterns associated with autism spectrum disorder in mouse models. As the R2C2 is a complete, lightweight end-to-end pipeline package with a graphical user interface and does not require end-user programming or heavy computation resources, it can be easily adopted in conventional neuroscience laboratories. By enabling effective auto-labeling of fine animal actions, R2C2 will facilitate studies aiming to uncover the neural mechanisms driving behavioral modulations.