Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions

TLDR

Dynamic texture extends texture into the temporal domain and has attracted growing research interest. The study proposes a novel dynamic texture recognition method and explores its simplifications and extensions to facial image analysis. The method models textures with volume local binary patterns (VLBP), uses co‑occurrences on three orthogonal planes (LBP‑TOP) for computational simplicity, and incorporates a block‑based approach to capture local spatial information in facial expressions. Experiments on DynTex, MIT, and Cohn‑Kanade databases show that VLBP and LBP‑TOP outperform prior techniques, with the block‑based method achieving excellent results and offering local processing, gray‑scale robustness, and simple computation.

Abstract

Dynamic texture (DT) is an extension of texture to the temporal domain. Description and recognition of DTs have attracted growing attention. In this paper, a novel approach for recognizing DTs is proposed and its simplifications and extensions to facial image analysis are also considered. First, the textures are modeled with volume local binary patterns (VLBP), which are an extension of the LBP operator widely used in ordinary texture analysis, combining motion and appearance. To make the approach computationally simple and easy to extend, only the co-occurrences of the local binary patterns on three orthogonal planes (LBP-TOP) are then considered. A block-based method is also proposed to deal with specific dynamic events such as facial expressions in which local information and its spatial locations should also be taken into account. In experiments with two DT databases, DynTex and Massachusetts Institute of Technology (MIT), both the VLBP and LBP-TOP clearly outperformed the earlier approaches. The proposed block-based method was evaluated with the Cohn-Kanade facial expression database with excellent results. The advantages of our approach include local processing, robustness to monotonic gray-scale changes, and simple computation.