S3DRGF: Spatial 3-D Relational Geometric Features for 3-D Sign Language Representation and Recognition

Abstract

Locations, angles, edges, and surfaces are spatial joint features that were predominantly used for characterizing three-dimensional (3-D) skeletal data in human action recognition. Despite their demonstrated success, features described earlier find difficulty in representing a relational change among joint movements in 3-D space for classifying human actions. To characterize a relation between joints on 3-D skeleton, we propose spatial 3-D relational geometric features (S3DRGFs). S3DRGFs are calculated on a subset of four joints in a chronological order covering all joints on the skeleton. Each of these four joints shape into a polygon, that reshapes spatially and temporally with respect to the sign (action) in the 3-D video. Consequently, we construct the spatio-temporal features (S3DRGF) by computing the area and perimeter of these polygons. Accordingly, query 3-D sign (action) recognition process transforms the joint area and perimeter features (JAF and JPF) into global alignment kernels based on the computed similarity scores with the dataset features. The similarity scores from JAF and JPF kernels are averaged for recognition. The proposed framework has been tested on our own 3-D sign language dataset (BVC3DSL) and three other publicly available datasets: HDM05, CMU, and NTU RGBD skeletal data. The results show higher levels of accuracy in decoding 3-D sign language into text for building a 3-D model based sign language translator.