ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction

by   Zhengdi Yu, et al.

Reconstructing two hands from monocular RGB images is challenging due to frequent occlusion and mutual confusion. Existing methods mainly learn an entangled representation to encode two interacting hands, which are incredibly fragile to impaired interaction, such as truncated hands, separate hands, or external occlusion. This paper presents ACR (Attention Collaboration-based Regressor), which makes the first attempt to reconstruct hands in arbitrary scenarios. To achieve this, ACR explicitly mitigates interdependencies between hands and between parts by leveraging center and part-based attention for feature extraction. However, reducing interdependence helps release the input constraint while weakening the mutual reasoning about reconstructing the interacting hands. Thus, based on center attention, ACR also learns cross-hand prior that handle the interacting hands better. We evaluate our method on various types of hand reconstruction datasets. Our method significantly outperforms the best interacting-hand approaches on the InterHand2.6M dataset while yielding comparable performance with the state-of-the-art single-hand methods on the FreiHand dataset. More qualitative results on in-the-wild and hand-object interaction datasets and web images/videos further demonstrate the effectiveness of our approach for arbitrary hand reconstruction. Our code is available at


page 1

page 4

page 5

page 7

page 8


Interacting Attention Graph for Single Image Two-Hand Reconstruction

Graph convolutional network (GCN) has achieved great success in single h...

Reconstructing Interacting Hands with Interaction Prior from Monocular Images

Reconstructing interacting hands from monocular images is indispensable ...

Exploiting Spatial-Temporal Context for Interacting Hand Reconstruction on Monocular RGB Video

Reconstructing interacting hands from monocular RGB data is a challengin...

See You Soon: Decoupled Iterative Refinement Framework for Interacting Hands Reconstruction from a Single RGB Image

Reconstructing interacting hands from a single RGB image is a very chall...

LWA-HAND: Lightweight Attention Hand for Interacting Hand Reconstruction

Hand reconstruction has achieved great success in real-time applications...

Im2Hands: Learning Attentive Implicit Representation of Interacting Two-Hand Shapes

We present Implicit Two Hands (Im2Hands), the first neural implicit repr...

Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements

3D interacting hand reconstruction is essential to facilitate human-mach...

Code Repositories


🔥(CVPR 2023) ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction

view repo

Please sign up or login with your details

Forgot password? Click here to reset