Human Parsing with Contextualized Convolutional Neural Network
In this work, we address the human parsing task with a novel Contextualized Convolutional Neural Network (Co-CNN) architecture, which well integrates the cross-layer context, global image-level context, semantic edge context, within-super-pixel context and cross-super-pixel neighborhood context into a unified network. Given an input human image, Co-CNN produces the pixel-wise categorization in an end-to-end way. First, the cross-layer context is captured by our basic local-to-global-to-local structure, which hierarchically combines the global semantic information and the local fine details across different convolutional layers. Second, the global image-level label prediction is used as an auxiliary objective in the intermediate layer of the Co-CNN, and its outputs are further used for guiding the feature learning in subsequent convolutional layers to leverage the global image-level context. Third, semantic edge context is further incorporated into Co-CNN, where the high-level semantic boundaries are leveraged to guide pixel-wise labeling. Finally, to further utilize the local super-pixel contexts, the within-super-pixel smoothing and cross-super-pixel neighbourhood voting are formulated as natural sub-components of the Co-CNN to achieve the local label consistency in both training and testing process. Comprehensive evaluations on two public datasets well demonstrate the significant superiority of our Co-CNN over other state-of-the-arts for human parsing. In particular, the F-1 score on the large dataset <xref ref-type="bibr" rid="ref1">[1]</xref> reaches <inline-formula><tex-math notation="LaTeX">$81.72\,\text{percent}$</tex-math><alternatives> <inline-graphic xlink:href="lin-ieq1-2537339.gif"/></alternatives></inline-formula> by Co-CNN, significantly higher than <inline-formula><tex-math notation="LaTeX">$62.81\,\text{percent}$</tex-math><alternatives> <inline-graphic xlink:href="lin-ieq2-2537339.gif"/></alternatives></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$64.38\,\text{percent}$</tex-math><alternatives> <inline-graphic xlink:href="lin-ieq3-2537339.gif"/></alternatives></inline-formula> by the state-of-the-art algorithms, M-CNN <xref ref-type="bibr" rid="ref2">[2]</xref> and ATR <xref ref-type="bibr" rid="ref1">[1]</xref> , respectively. By utilizing our newly collected large dataset for training, our Co-CNN can achieve <inline-formula> <tex-math notation="LaTeX">$85.36\,\text{percent}$</tex-math><alternatives> <inline-graphic xlink:href="lin-ieq4-2537339.gif"/></alternatives></inline-formula> in F-1 score.