A dynamic light image enhancement algorithm using generative adversarial network for group activity recognition

Video-based or image-based human activity recognition (HAR) via machine learning algorithms helps track, detect, and categorize users' daily activities. It is usually formulated as specific research problems, such as fall detection, gait recognition, posture recognition, and gesture recognition. In practice, HAR is extended to a more generic group activity recognition (GAR) problem, focusing on multiple user (group) activities for various purposes, such as the recognition of social activities, autonomous driving, surveillance, and pedestrian crossing management. In this research, the formulation of the GAR problem aims to enhance the performance of the GAR model by tackling the research limitations in three aspects: (i) image quality may not always be guaranteed, resulting bias and inaccurate models; (ii) the working environments, in both indoor and outdoor, of the GAR model are dynamic, such as varying light conditions; and (iii) the number of people in the covered area may vary and the number of class labels for activities is large, leading to an extensive computing time to train a complex GAR model architecture. Therefore, a dynamic light image enhancement generative adversarial network has been proposed to enhance the image quality significantly. A multi-input image-enhanced generative adversarial network (MIIEGAN) has been proposed for generating high-quality additional training images, and a guided asymmetric depthwise separable convolution (GA-DSC) contributed to the model complexity and performance. These algorithms were evaluated in dynamic light environments to investigate their robustness and performance in varying light conditions that meet the nature of GAR in indoor and outdoor environments. Two benchmark datasets (The Volleyball Dataset and The Collective Dataset) were selected for the performance evaluation and comparison. Five backbones (AlexNet, VGG-16, Inception-v3, ResNet-50, and EfficientNetV2) were chosen as deep learning models for analysis. Nine existing methods were compared. Five variants of GANs and five variants of convolutional techniques were compared in the ablation experiments. Through experiments and ablation studies, the proposed work outperforms existing works (0.931-9.29% and 1.05-9.83%), five variants of GANs (on average 2.71% and 2.62%), and five variants of convolutional techniques (on average 1.91% and 1.75%).