Monocular 3D palm posture estimation based on feature-points robust against finger motion

Abstract

The usability of wearable augmented reality (AR) systems would improve by letting users arbitrarily display virtual information on their palm and simultaneously manipulate it as tablet computers or smartphones. To realize such interaction between users and virtual information, we aim to robustly estimate 3-D palm posture against finger motion. This is based on the assumption that finger motion is separately estimated from palm posture and applied to manipulation of displayed information. In addition, the capability of electric sources, sensors, and processors are very limited in wearable computers. For this reason, by using a monocular camera and estimating palm posture from only a few image feature-points, we achieve an efficient estimation that satisfies real-time constraint on wearable computers. The accuracy and the robustness of our method are demonstrated by qualitative and quantitative comparison with a widely-used cardboard maker. Additionally, we confirmed that our method is run on a mobile computer at the average of 12.44 msec per frame.