face.evoLVe: High-Performance Face Recognition Library based on PaddlePaddle & PyTorch

• Evolve to be more comprehensive, effective and efficient for face related analytics & applications! (WeChat News)
• About the name:
  • "face" means this repo is dedicated for face related analytics & applications.
  • "evolve" means unleash your greatness to be better and better. "LV" are capitalized to acknowledge the nurturing of Learning and Vision (LV) group, Nation University of Singapore (NUS).
• This work was done during Jian Zhao served as a short-term "Texpert" Research Scientist at Tencent FiT DeepSea AI Lab, Shenzhen, China.

|Author|Jian Zhao| |:---:|:---:| |Homepage|https://zhaoj9014.github.io|

---

License

The code of face.evoLVe is released under the MIT License.

---

News

:white_check_mark: CLOSED 02 September 2021: ~~Baidu PaddlePaddle officially merged face.evoLVe to faciliate researches and applications on face-related analytics (Official Announcement).~~

:white_check_mark: CLOSED 03 July 2021: ~~Provides training code for the paddlepaddle framework.~~

:white_check_mark: CLOSED 04 July 2019: ~~We will share several publicly available datasets on face anti-spoofing/liveness detection to facilitate related research and analytics.~~

:white_check_mark: CLOSED 07 June 2019: ~~We are training a better-performing IR-152 model on MS-Celeb-1M_Align_112x112, and will release the model soon.~~

:white_check_mark: CLOSED 23 May 2019: ~~We share three publicly available datasets to facilitate research on heterogeneous face recognition and analytics. Please refer to Sec. Data Zoo for details.~~

:white_check_mark: CLOSED 23 Jan 2019: ~~We share the name lists and pair-wise overlapping lists of several widely-used face recognition datasets to help researchers/engineers quickly remove the overlapping parts between their own private datasets and the public datasets. Please refer to Sec. Data Zoo for details.~~

:white_check_mark: CLOSED 23 Jan 2019: ~~The current distributed training schema with multi-GPUs under PyTorch and other mainstream platforms parallels the backbone across multi-GPUs while relying on a single master to compute the final bottleneck (fully-connected/softmax) layer. This is not an issue for conventional face recognition with moderate number of identities. However, it struggles with large-scale face recognition, which requires recognizing millions of identities in the real world. The master can hardly hold the oversized final layer while the slaves still have redundant computation resource, leading to small-batch training or even failed training. To address this problem, we are developing a highly-elegant, effective and efficient distributed training schema with multi-GPUs under PyTorch, supporting not only the backbone, but also the head with the fully-connected (softmax) layer, to facilitate high-performance large-scale face recognition. We will added this support into our repo.~~

:white_check_mark: CLOSED 22 Jan 2019: ~~We have released two feature extraction APIs for extracting features from pre-trained models, implemented with PyTorch build-in functions and OpenCV, respectively. Please check ./util/extract_feature_v1.py and ./util/extract_feature_v2.py.~~

:white_check_mark: CLOSED 22 Jan 2019: ~~We are fine-tuning our released IR-50 model on our private Asia face data, which will be released soon to facilitate high-performance Asia face recognition.~~

:white_check_mark: CLOSED 21 Jan 2019: ~~We are training a better-performing IR-50 model on MS-Celeb-1M_Align_112x112, and will replace the current model soon.~~

---

Contents

• Introduction
• Pre-Requisites
• Usage
• Face Alignment
• Data Processing
• Training and Validation
• Data Zoo
• Model Zoo
• Achievement
• Acknowledgement
• Donation
• Citation

---

face.evoLVe for High-Performance Face Recognition

Introduction

:information_desk_person:

<img src="https://github.com/ZhaoJ9014/face.evoLVe/blob/master/disp/Fig1.png" width="450px"/> <img src="https://github.com/ZhaoJ9014/face.evoLVe/blob/master/disp/Fig17.png" width="400px"/>

• This repo provides a comprehensive face recognition library for face related analytics & applications, including face alignment (detection, landmark localization, affine transformation, etc.), data processing (e.g., augmentation, data balancing, normalization, etc.), various backbones (e.g., ResNet, IR, IR-SE, ResNeXt, SE-ResNeXt, DenseNet, LightCNN, MobileNet, ShuffleNet, DPN, etc.), various losses (e.g., Softmax, Focal, Center, SphereFace, CosFace, AmSoftmax, ArcFace, Triplet, etc.) and bags of tricks for improving performance (e.g., training refinements, model tweaks, knowledge distillation, etc.).
• The current distributed training schema with multi-GPUs under PyTorch and other mainstream platforms parallels the backbone across multi-GPUs while relying on a single master to compute the final bottleneck (fully-connected/softmax) layer. This is not an issue for conventional face recognition with moderate number of identities. However, it struggles with large-scale face recognition, which requires recognizing millions of identities in the real world. The master can hardly hold the oversized final layer while the slaves still have redundant computation resource, leading to small-batch training or even failed training. To address this problem, this repo provides a highly-elegant, effective and efficient distributed training schema with multi-GPUs under PyTorch, supporting not only the backbone, but also the head with the fully-connected (softmax) layer, to facilitate high-performance large-scale face recognition.
• All data before & after alignment, source codes and trained models are provided.
• This repo can help researchers/engineers develop high-performance deep face recognition models and algorithms quickly for practical use and deployment.

---

Pre-Requisites

:cake:

• Linux or macOS
• Python 3.7 (for training & validation) and Python 2.7 (for visualization w/ tensorboardX)
• PyTorch 1.0 (for traininig & validation, install w/ pip install torch torchvision)
• MXNet 1.3.1 (optional, for data processing, install w/ pip install mxnet-cu90)
• TensorFlow 1.12 (optional, for visualization, install w/ pip install tensorflow-gpu)
• tensorboardX 1.6 (optional, for visualization, install w/ pip install tensorboardX)
• OpenCV 3.4.5 (install w/ pip install opencv-python)
• bcolz 1.2.0 (install w/ pip install bcolz)

While not required, for optimal performance it is highly recommended to run the code using a CUDA enabled GPU. We used 4-8 NVIDIA Tesla P40 in parallel.

---

Usage

:orange_book:

• Clone the repo: git clone https://github.com/ZhaoJ9014/face.evoLVe.PyTorch.git.
• mkdir data checkpoint log at appropriate directory to store your train/val/test data, checkpoints and training logs.
• Prepare your train/val/test data (refer to Sec. Data Zoo for publicly available face related databases), and ensure each database folder has the following structure:

  ./data/db_name/
          -> id1/
              -> 1.jpg
              -> ...
          -> id2/
              -> 1.jpg
              -> ...
          -> ...
              -> ...
              -> ...
  

• Refer to the codes of corresponding sections for specific purposes.

---

Face Alignment

:triangular_ruler:

<img src="https://github.com/ZhaoJ9014/face.evoLVe/blob/master/disp/Fig2.png" width="900px"/> <img src="https://github.com/ZhaoJ9014/face.evoLVe/blob/master/disp/Fig3.png" width="500px"/>

• This section is based on the work of MTCNN.
• Folder: ./align
• Face detection, landmark localization APIs and visualization toy example with ipython notebook:

  from PIL import Image
  from detector import detect_faces
  from visualization_utils import show_results
  
  img = Image.open('some_img.jpg') # modify the image path to yours
  bounding_boxes, landmarks = detect_faces(img) # detect bboxes and landmarks for all faces in the image
  show_results(img, bounding_boxes, landmarks) # visualize the results
  

• Face alignment API (perform face detection, landmark localization and alignment with affine transformations on a whole database folder source_root with the directory structure as demonstrated in Sec. Usage, and store the aligned results to a new folder dest_root with the same directory structure):

  python face_align.py -source_root [source_root] -dest_root [dest_root] -crop_size [crop_size]
  
  # python face_align.py -source_root './data/test' -dest_root './data/test_Aligned' -crop_size 112
  

• For macOS users, there is no need to worry about *.DS_Store files which may ruin your data, since they will be automatica