List of facial expression databases

A facial expression database is a collection of images or video clips with facial expressions of a range of emotions. Well-annotated (emotion-tagged) media content of facial behavior is essential for training, testing, and validation of algorithms for the development of expression recognition systems. The emotion annotation can be done in discrete emotion labels or on a continuous scale. Most of the databases are usually based on the basic emotions theory (by Paul Ekman) which assumes the existence of six discrete basic emotions (anger, fear, disgust, surprise, joy, sadness). However, some databases include the emotion tagging in continuous arousal-valence scale.

In posed expression databases, the participants are asked to display different basic emotional expressions, while in spontaneous expression database, the expressions are natural. Spontaneous expressions differ from posed ones remarkably in terms of intensity, configuration, and duration. Apart from this, synthesis of some AUs are barely achievable without undergoing the associated emotional state. Therefore, in most cases, the posed expressions are exaggerated, while the spontaneous ones are subtle and differ in appearance.

Many publicly available databases are categorized here.^[1][2] Here are some details of the facial expression databases.

Database	Facial expression	Number of Subjects	Number of images/videos	Gray/Color	Resolution, Frame rate	Ground truth	Type
FERG-3D-DB (Facial Expression Research Group 3D Database) for stylized characters [3]	angry, disgust, fear, joy, neutral, sad, surprise	4	39574 annotated examples	Color		Emotion labels	Frontal pose
Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) [4]	Speech: Calm, happy, sad, angry, fearful, surprise, disgust, and neutral. Song: Calm, happy, sad, angry, fearful, and neutral. Each expression at two levels of emotional intensity.	24	7356 video and audio files	Color	1280x720 (720p)	Facial expression labels Ratings provided by 319 human raters	Posed
Extended Cohn-Kanade Dataset (CK+)[5]	neutral, sadness, surprise, happiness, fear, anger, contempt and disgust	123	593 image sequences (327 sequences having discrete emotion labels)	Mostly gray	640* 490	Facial expression labels and FACS (AU label for final frame in each image sequence)	Posed; spontaneous smiles
Japanese Female Facial Expressions (JAFFE)[6]	neutral, sadness, surprise, happiness, fear, anger, and disgust	10	213 static images	Gray	256* 256	Facial expression label	Posed
MMI Database[7]		43	1280 videos and over 250 images	Color	720* 576	AU label for the image frame with apex facial expression in each image sequence	Posed and Spontaneous
Belfast Database[8]	Set 1 (disgust, fear, amusement, frustration, surprise)	114	570 video clips	Color	720*576		Natural Emotion
	Set 2 (disgust, fear, amusement, frustration, surprise, anger, sadness)	82	650 video clips	Color
	Set 3 (disgust, fear, amusement)	60	180 video clips	Color	1920*1080
Indian Semi-Acted Facial Expression Database (iSAFE)[9]	Happy, Sad, Fear, Surprise, Angry, Neutral, Disgust	44	395 clips	Color	1920x1080 (60 fps)	Emotion labels	Spontaneous
DISFA[10]	-	27	4,845 video frames	Color	1024*768; 20 fps	AU intensity for each video frame (12 AUs)	Spontaneous
Multimedia Understanding Group (MUG)[11]	neutral, sadness, surprise, happiness, fear, anger, and disgust	86	1462 sequences	Color	896*896, 19fps	Emotion labels	Posed
Indian Spontaneous Expression Database (ISED)[12]	sadness, surprise, happiness, and disgust	50	428 videos	Color	1920* 1080, 50 fps	Emotion labels	Spontaneous
Radboud Faces Database (RaFD)[13]	neutral, sadness, contempt, surprise, happiness, fear, anger, and disgust	67	Three different gaze directions and five camera angles (8*67*3*5=8040 images)	Color	681*1024	Emotion labels	Posed
Oulu-CASIA NIR-VIS database	surprise, happiness, sadness, anger, fear and disgust	80	three different illumination conditions: normal, weak and dark (total 2880 video sequences)	Color	320×240		Posed
FERG (Facial Expression Research Group Database)-DB[14] for stylized characters	angry, disgust, fear, joy, neutral, sad, surprise	6	55767	Color	768x768	Emotion labels	Frontal pose
AffectNet[15]	neutral, happy, sad, surprise, fear, disgust, anger, contempt		~450,000 manually annotated ~ 500,000 automatically annotated	Color	Various	Emotion labels, valence, arousal	Wild setting
IMPA-FACE3D[16]	neutral frontal, joy, sadness, surprise, anger, disgust, fear, opened, closed, kiss, left side, right side, neutral sagittal left, neutral sagittal right, nape and forehead (acquired sometimes)	38	534 static images	Color	640X480	Emotion labels	Posed
FEI Face Database	neutral,smile	200	2800 static images	Color	640X480	Emotion labels	Posed
Aff-Wild[17][18]	valence and arousal	200	~1,250,000 manually annotated	Color	Various (average = 640x360)	Valence, Arousal	In-the-Wild setting
Aff-Wild2[19][20]	neutral, happiness, sadness, surprise, fear, disgust, anger + valence-arousal + action units 1,2,4,6,12,15,20,25	458	~2,800,000 manually annotated	Color	Various (average = 1030x630)	Valence, Arousal, 7 basic expressions, action units for each video frame	In-the-Wild setting
Real-world Affective Faces Database (RAF-DB)[21][22]	6 classes of basic emotions (Surprised, Fear, Disgust, Happy, Sad, Angry) plus Neutral and 12 classes of compound emotions (Fearfully Surprised, Fearfully Disgusted, Sadly Angry, Sadly Fearful, Angrily Disgusted, Angrily Surprised, Sadly Disgusted, Disgustedly Surprised, Happily Surprised, Sadly Surprised, Fearfully Angry, Happily Disgusted)		29672 annotated examples	Color	Various for original dataset and 100x100 for aligned dataset	Emotion labels	Posed and Spontaneous