When human faces can be resolved sufficiently well, facial emotions can be checked to see whether these again are compatible with what one expects from their movements and posture in the observational and locational context.
Detection of faces
Face detection is a primary step in many applications such as face recognition, video surveillance, human computer interface, and expression recognition. Many existing detection techniques suffer under scale variation, pose variation (frontal vs. profile), illumination changes, and complex backgrounds.
In our investigations, we use a robust and efficient method for face detection in colour images. Skin colour segmentation and edge detection are employed to separate all non-face regions from the candidate faces. Primitive shape features are then used to decide which of the candidate regions actually correspond to a face.
The advantage of our method is its ability to achieve a high detection rate under varying conditions (pose, scale,...) with low computational cost.
Tracking of face motion
We have proposed a hierarchical face and gaze tracking by wise-combination of Appearance-based Trackers (ABT), which estimates predefined facial features in monocular video sequences. A non-occluded facial texture was used to estimate the eyelid position for any kind of blinking. Also, we extended the appearance model to track the iris motion, while achieving correct adaptation after saccade motion or eyelid occlusions.
We have also combined 3D head pose, eyebrows, lips, eyelids and gazes in a hierarchical framework for real-time tracking. Unlike previous approaches related to eyelid and iris tracking, we avoided edge detections, colour information, and other thresholding techniques.
Interpretation of facial expressions
Facial expression analysis is an interesting subject due to the relevance of the expressions on human emotions. We proposed a Case-based Reasoning (CBR) classification procedure for facial expression analysis which achieves a high recognition rate by assessing confidence for the estimated classifications.
As advantages, the training process is achieved with spontaneous facial expressions which are more natural than the forced expressions currently being used in the literature. We included the eye motion to evaluate their relevance on the facial expressions, being the first step for trust and deceit analysis. Lastly, we improved the quality of the estimations by evaluating several confidence measures.