Neural network model for video-based face recognition with frames quality assessment

This paper addresses a problem of video-based face recognition. We propose a new neural network model that uses an input set of facial images of a person to produce a compact, fixed-dimension descriptor. Our model is composed of two modules. The feature embedding module maps each image onto a feature vector, while the face quality assessment module estimates the utility of each facial image. These feature vectors are weighted based on their utility estimations, resulting in the image set feature representation. During visual analysis we found that our model learns to use more information from high-quality face images and less information from blurred or occluded images. The experiments on YouTube Faces and Janus Benchmark A (IJB-A) datasets show that the proposed feature aggregation method based on face quality assessment consistently outperforms naïve aggregation methods.