Neural networks for intelligent multimedia processing

Abstract

This paper reviews key attributes of neural processing essential to intelligent multimedia processing (IMP). The objective is to show why neural networks (NNs) are a core technology for the following multimedia functionalities: (1) efficient representations for audio/visual information, (2) detection and classification techniques, (3) fusion of multimodal signals, and (4) multimodal conversion and synchronization. It also demonstrates how the adaptive NN technology presents a unified solution to a broad spectrum of multimedia applications. As substantiating evidence, representative examples where NNs are successfully applied to IMP applications are highlighted. The examples cover a broad range, including image visualization, tracking of moving objects, image/video segmentation, texture classification, face-object detection/recognition, audio classification, multimodal recognition, and multimodal lip reading.