Single-to-Dual-View Adaptation for Egocentric 3D Hand Pose Estimation

The pursuit of accurate 3D hand pose estimation stands as a keystone forunderstanding human activity in the realm of egocentric vision. The majority ofexisting estimation methods still rely on single-view images as input, leadingto potential limitations, e.g., limited field-of-view and ambiguity in depth.To address these problems, adding another camera to better capture the shape ofhands is a practical direction. However, existing multi-view hand poseestimation methods suffer from two main drawbacks: 1) Requiring multi-viewannotations for training, which are expensive. 2) During testing, the modelbecomes inapplicable if camera parameters/layout are not the same as those usedin training. In this paper, we propose a novel Single-to-Dual-view adaptation(S2DHand) solution that adapts a pre-trained single-view estimator to dualviews. Compared with existing multi-view training methods, 1) our adaptationprocess is unsupervised, eliminating the need for multi-view annotation. 2)Moreover, our method can handle arbitrary dual-view pairs with unknown cameraparameters, making the model applicable to diverse camera settings.Specifically, S2DHand is built on certain stereo constraints, includingpair-wise cross-view consensus and invariance of transformation between bothviews. These two stereo constraints are used in a complementary manner togenerate pseudo-labels, allowing reliable adaptation. Evaluation results revealthat S2DHand achieves significant improvements on arbitrary camera pairs underboth in-dataset and cross-dataset settings, and outperforms existing adaptationmethods with leading performance. Project page:https://github.com/MickeyLLG/S2DHand.