Facial action units detection under pose variations using deep regions learning
A set of facial Action Units (AUs) is activated due to the movement of facial muscles in response to a person's internal emotion. The activated facial action units appear around sparse regions on the face e.g., the mouth and eyes. This group occurrence of AUs reveals the semantic relationships among them. These relationships should be considered in the detection process of AUs. Therefore, we propose an approach that learns these semantic relationships using a multi-label deep learning architecture. The sparse patches of the AUs are used in the proposed approach instead of the whole facial region. To handle these sparse patches, we propose a region-based network instead of the well-known convolutional network and the locally connected network. Moreover, unlike the current approaches, which define facial regions using a uniform grid, the proposed region-based architecture, define patches around facial landmarks. This overcomes the region displacement problem of the uniform grid. Usually, the AUs classification suffers from the high skewness factor problem. To overcome this problem, we use a weighted loss function. We conduct our experiments on two standard benchmarks: BP4D and FERA17. f1 score, the area under the ROC (AROC) and the area under the precision-recall curve (APR) are used as performance measures. Compared to the standard convolutional layer, the proposed patch-based layer is more effective in capturing the required structural features of the face and learns the correlations among AUs under pose variations. Also, it has been shown that the proposed approach outperforms the state-of-the-art methods.