CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction

TLDR

Modeling the hand‑object interaction requires not only estimating the pose but also accounting for contact, yet simultaneous pose estimation and contact modeling remains underexplored despite progress in separate hand and object estimation. We aim to introduce a Contact Potential Field (CPF) and a hybrid learning‑fitting framework (MIHO) to explicitly model hand‑object interaction. CPF treats each contacting hand‑object vertex pair as a spring‑mass system, forming a potential field whose minimal elastic energy corresponds to the grasp configuration. Experiments on two standard benchmarks show that our method achieves state‑of‑the‑art reconstruction metrics and produces more physically plausible hand‑object poses even when ground‑truth data exhibit severe interpenetration or disjointedness. Code is available at https://github.com/lixiny/CPF.

Abstract

Modeling the hand-object (HO) interaction not only requires estimation of the HO pose, but also pays attention to the contact due to their interaction. Significant progress has been made in estimating hand and object separately with deep learning methods, simultaneous HO pose estimation and contact modeling has not yet been fully explored. In this paper, we present an explicit contact representation namely Contact Potential Field (CPF), and a learning-fitting hybrid framework namely MIHO to Modeling the Interaction of Hand and Object. In CPF, we treat each contacting HO vertex pair as a spring-mass system. Hence the whole system forms a potential field with minimal elastic energy at the grasp position. Extensive experiments on the two commonly used benchmarks have demonstrated that our method can achieve state-of-the-art in several reconstruction metrics, and allow us to produce more physically plausible HO pose even when the ground-truth exhibits severe interpenetration or disjointedness. Our code is available at https://github.com/lixiny/CPF.