Learning deep representations from 2D images and 3D point clouds for gait recognition

Abstract

Gait recognition, a biometric technology aimed at identifying individuals from a distance, holds great potential for applications in long-range pedestrian recognition scenarios. However, despite recent advancements in deep learning techniques, the challenge of learning robust representations capable of adapting to realistic environments still remains. In real-world scenarios, gait can exhibit significant diversity due to various factors such as camera perspectives, clothing variations, and environmental conditions. Existing approaches mainly rely on image-based modalities, limiting their applicability in various conditions. This thesis addresses the limitations of current gait recognition methods and explores novel approaches to enhance their robustness. The research focuses on image-based gait recognition and investigating LiDAR-based gait recognition in outdoor environments.

First, this thesis tackles the challenge of fine-grained gait representation learning within existing image-based gait recognition methods. Existing approaches utilize global representation learning or part-based local representation learning, but these methods still neglect partial features around local parts of the human body. Unlike previous methods, we propose a novel mask-based regularization technique, ReverseMask, to learn fine-grained representations for gait recognition from 2D images. ReverseMask injects perturbations into feature maps, aiding convolutional architectures in learning discriminative representations and improving generalization. Experimental results on popular datasets validate the effectiveness of this regularization method.

Second, this thesis addresses the problem of gait recognition in an outdoor environment. Although conventional gait recognition research predominantly relies on image-based modalities, these methods often overlook crucial human 3D structural information and are constrained by visual ambiguity. Existing 2D gait recognition limits the application of gait recognition in the 3D wild world. Thus, we introduce a large-scale LiDAR-based gait recognition benchmark and present a novel 3D gait recognition framework. We demonstrate the superiority of LiDAR-based methods over traditional silhouette-based approaches in outdoor environments. Furthermore, we propose LidarGait, a simple yet effective 3D gait recognition method. LidarGait projects sparse point clouds into depth maps, enabling the learning of representations enriched with 3D geometry information.

Furthermore, this thesis addresses the challenge of 3D representation learning from point clouds. While 3D gait recognition is introduced earlier in this thesis, the task of extracting 3D gait features directly from sparse and unordered data remains unresolved. Through systematic analysis, we identify locality and size awareness as crucial factors for effective 3D gait recognition. Consequently, we introduce S$^3$GaitNet, which incorporates an advanced size-aware learning mechanism to enhance the discriminative ability of learned features. Extensive experiments validate that our proposed methods effectively learn locality and size awareness, leading to enhanced generalizability in 3D gait recognition tasks.

In summary, this thesis contributes novel techniques for improving gait recognition in both image-based and LiDAR-based modalities. The proposed methods offer improved performance and robustness, addressing key challenges in real-world scenarios. This work not only advances the field of gait recognition but also opens avenues for future research in 3D gait recognition applications.