The backbone network pre-trained by LiftedCL can be transferred to various human-centric downstream tasks including human pose estimation, human shape recovery and human parsing.

Abstract

Human-centric perception targets for understanding human body pose, shape and segmentation. Pre-training the model on large-scale datasets and fine-tuning it on specific tasks has become a well-established paradigm in human-centric perception. Recently, self-supervised learning methods have re-investigated contrastive learning to achieve superior performance on various downstream tasks. When handling human-centric perception, there still remains untapped potential since 3D human structure information is neglected during the task-agnostic pre-training. In this paper, we propose the Lifting Contrastive Learning (LiftedCL) to obtain 3D-aware human-centric representations which absorb 3D human structure information. In particular, to induce the learning process, a set of 3D skeletons is randomly sampled by resorting to 3D human kinematic prior. With this set of generic 3D samples, 3D human structure information can be learned into 3D-aware representations through adversarial learning. Empirical results demonstrate that LiftedCL outperforms state-of-the-art self-supervised methods on four human-centric downstream tasks, including 2D and 3D human pose estimation (0.4% mAP and 1.8 mm MPJPE improvement on COCO 2D pose estimation and Human3.6M 3D pose estimation), human shape recovery and human parsing.

Paper

@inproceedings{
                chen2023liftedcl,
                title={Lifted{CL}: Lifting Contrastive Learning for Human-Centric Perception},
                author={Ziwei Chen and Qiang Li and Xiaofeng Wang and Wankou Yang},
                booktitle={The Eleventh International Conference on Learning Representations },
                year={2023},
                url={https://openreview.net/forum?id=WHlt5tLz12T}
                }
            

Code and Weights

Acknowledgements

This work was supported by the National Natural Science Foundation of China under No. 62276061. Webpage template.