Monocular 3D Facial Information Retrieval for Automated Facial Expression Analysis

Meshia Cédric Oveneke, Isabel Gonzalez, Weiyi Wang, Dongmei Jiang, Hichem Sahli

Research output: Chapter in Book/Report/Conference proceedingConference paper

1 Citation (Scopus)

Abstract

Understanding social signals is a very important aspect of human communication and interaction and has therefore attracted increased attention from various research areas. Among the different types of social signals, particular attention has been paid to facial expression of emotions and its automated analysis from image sequences. Automated facial expression analysis is a very challenging task due to the complex three-dimensional deformation and motion of the face associated to the facial expressions and the loss of 3D information during the image formation process. As a consequence, retrieving 3D spatio-temporal facial information from image sequences is essential for automated facial expression analysis. In this paper, we propose a framework for retrieving three-dimensional facial structure, motion and spatio-temporal features from monocular image sequences. First, we estimate monocular 3D scene flow by retrieving the facial structure using shape-from-shading (SFS) and combine it with 2D optical flow. Secondly, based on the retrieved structure and motion of the face, we extract spatio-temporal features for automated facial expression analysis. Experimental results illustrate the potential of the proposed 3D facial information retrieval framework for facial expression analysis, i.e. facial expression recognition and facial action-unit recognition on a benchmark dataset. This paves the way for future research on monocular 3D facial expression analysis.
Original languageEnglish
Title of host publicationInternational Conference on Affective Computing and Intelligent Interaction (ACII) 2015
Place of PublicationXi'an, China
PublisherIEEE
Pages623 - 629
DOIs
Publication statusPublished - 2015
EventAffective Computing and Intelligent Interaction (ACII), 2015 International Conference on - China, Xi'an, China
Duration: 21 Sep 201524 Sep 2015

Conference

ConferenceAffective Computing and Intelligent Interaction (ACII), 2015 International Conference on
CountryChina
CityXi'an
Period21/09/1524/09/15

Keywords

  • Facial Expression Analysis
  • Shape-from- Shading
  • 3D Scene Flow Estimation
  • Bag-of-Visual-Words
  • Multiple Instance Learning

Fingerprint Dive into the research topics of 'Monocular 3D Facial Information Retrieval for Automated Facial Expression Analysis'. Together they form a unique fingerprint.

Cite this