HRHPE: FRoI guides heterogeneous relationship representation learning for precise head pose estimation

Abstract

<p>The ability to effectively detect head positions has raised concerns in the field of computer vision. However, head pose estimation (HPE) is prone to problems, such as extreme angles, occlusion, and lighting. To effectively address this critical missing information gap in the field of HPE, we propose a heterogeneous relationship guided learning method for the representation of transannular layers that effectively captures and uses “Facial Regions of Interest” (FRoI) and heterogeneous relationships through FRoI morphable modeling. Two key observations are revealed: 1) the saliency of the facial regions of interest; and 2) the heterogeneous relationships between the adjacent postures. On this basis, three modules are proposed, namely Region Feature Generation (RFG), Hierarchical Structure Modeling (HSM) and Heterogeneous Relationship Mining (HRM). In particular, we introduce a regional attention mechanism in the RFG, which assigns a higher weighting to FRoI. In HSM, the concept of the “Rugby style” design is proposed as a model for the cross-layer structure. In HRM, we use the Transformer to explore the interdependencies between facial regions and semantic relationships in rotation of the head. Experiments with three real HPE datasets (300 W_LP, AFLW2000 and BIWI) show that our HRHPE is more efficient than the state-of-the-art methods.</p>