Face analysis is a big challenging branch in computer vision and Biometric and there are several datasets related to the face and its challenges like age, orientation, skin color, illumination, pose, expressions, appearance, etc. So, researchers almost using different datasets related to the challenges, the face recognition homepage is a website contain several face datasets which is nice to find and compare datasets and recently published related papers.

Some of the famous datasets are:

The Color FERET Database, USA

Multi-PIE

The Yale Face Database

Cohn-Kanade AU Coded Facial Expression Database

NIST Mugshot Identification Database

The AR Face Database, The Ohio State University, USA

The Sheffield Face Database (previously: The UMIST Face Database)

Georgia Tech Face Database

Labeled Faces in the Wild

Labeled Faces in the Wild-a (LFW-a)

YouTube Faces Database

and so on ...