FistPointer

Abstract

We present a target selection technique using mid-air interaction behind a smartphone for mobile virtual reality (VR) environments. This is realized by using a smartphone as a head-mounted display with a cardboard viewer. Our technique displays a pointer on the screen corresponding to the position of the hand, which is detected by the built-in back camera of the smartphone. The user can move the pointer by moving the hand in a thumbs-up position. The user can also select a target by folding the thumb, similarly to pushing a joystick button. Our technique can be implemented using only a smartphone with a built-in back camera; therefore, it is easy to apply to target selection in mobile VR environments. To test the feasibility of the design, we prototyped a proof-of-concept implementation. Furthermore, we developed a game using our technique and investigated user impressions.