facetorch

User Guide, Documentation, ChatGPT facetorch guide

Docker Hub (GPU)

Facetorch is a Python library designed for facial detection and analysis, leveraging the power of deep neural networks. Its primary aim is to curate open-source face analysis models from the community, optimize them for high performance using TorchScript, and integrate them into a versatile face analysis toolkit. The library offers the following key features:

Customizable Configuration: Easily configure your setup using Hydra and its powerful OmegaConf capabilities.
Reproducible Environments: Ensure reproducibility with tools like conda-lock for dependency management and Docker for containerization.
Accelerated Performance: Enjoy enhanced performance on both CPU and GPU with TorchScript optimization.
Simple Extensibility: Extend the library by uploading your model file to Hugging Face Hub (previously Google Drive) and adding a corresponding configuration YAML file to the repository.

Facetorch provides an efficient, scalable, and user-friendly solution for facial analysis tasks, catering to developers and researchers looking for flexibility and performance.

Please use this library responsibly and with caution. Adhere to the European Commission's Ethics Guidelines for Trustworthy AI to ensure ethical and fair usage. Keep in mind that the models may have limitations and potential biases, so it is crucial to evaluate their outputs critically and consider their impact.

Install

PyPI

pip install facetorch

Conda

conda install -c conda-forge facetorch

Usage

Prerequisites

Docker Compose provides an easy way of building a working facetorch environment with a single command.

Run docker example

CPU: docker compose run facetorch python ./scripts/example.py
GPU: docker compose run facetorch-gpu python ./scripts/example.py analyzer.device=cuda

Check data/output for resulting images with bounding boxes and facial 3D landmarks.

(Apple Mac M1) Use Rosetta 2 emulator in Docker Desktop to run the CPU version.

Configure

The project is configured by files located in conf with the main file: conf/config.yaml. One can easily add or remove modules from the configuration.

Components

FaceAnalyzer is the main class of facetorch as it is the orchestrator responsible for initializing and running the following components:

Reader - reads the image and returns an ImageData object containing the image tensor.
Detector - wrapper around a neural network that detects faces.
Unifier - processor that unifies sizes of all faces and normalizes them between 0 and 1.
Predictor dict - set of wrappers around neural networks trained to analyze facial features.
Utilizer dict - set of wrappers around any functionality that requires the output of neural networks e.g. drawing bounding boxes or facial landmarks.

Structure

analyzer
    ├── reader
    ├── detector
    ├── unifier
    └── predictor
            ├── embed
            ├── verify
            ├── fer
            ├── au
            ├── va
            ├── deepfake
            └── align
    └── utilizer
            ├── align
            ├── draw
            └── save

Models

Detector

|     model     |   source  |   params  |   license   | version |
| ------------- | --------- | --------- | ----------- | ------- |
|   RetinaFace  |  biubug6  |   27.3M   | MIT license |    1    |

biubug6

Predictor

Facial Representation Learning (embed)

|       model       |   source   |  params |   license   | version |  
| ----------------- | ---------- | ------- | ----------- | ------- |
|  ResNet-50 VGG 1M |  1adrianb  |  28.4M  | MIT license |    1    |

1adrianb
- code: unsupervised-face-representation
- paper: Bulat et al. - Pre-training strategies and datasets for facial representation learning
- Note: include_tensors needs to be True in order to include the model prediction in Prediction.logits

Face Verification (verify)

|       model      |   source    |  params  |      license       | version |  
| ---------------- | ----------- | -------- | ------------------ | ------- |
|    MagFace+UNPG  | Jung-Jun-Uk |   65.2M  | Apache License 2.0 |    1    |
|  AdaFaceR100W12M |  mk-minchul |    -     |     MIT License    |    2    |

Jung-Jun-Uk
- code: UNPG
- paper: Jung et al. - Unified Negative Pair Generation toward Well-discriminative Feature Space for Face Recognition
- (FAR=0.01)
- Note: include_tensors needs to be True in order to include the model prediction in Prediction.logits
mk-minchul
- code: AdaFace
- paper: Kim et al. - AdaFace: Quality Adaptive Margin for Face Recognition
- <
- <
- < badges represent models trained on smaller WebFace 4M dataset
- Note: include_tensors needs to be True in order to include the model prediction in Prediction.logits

Facial Expression Recognition (fer)

|       model       |      source    |  params  |       license      | version |  
| ----------------- | -------------- | -------- | ------------------ | ------- |
| EfficientNet B0 7 | HSE-asavchenko |    4M    | Apache License 2.0 |    1    |
| EfficientNet B2 8 | HSE-asavchenko |   7.7M   | Apache License 2.0 |    2    |

HSE-asavchenko
- code: face-emotion-recognition
- paper: Savchenko - Facial expression and attributes recognition based on multi-task learning of lightweight neural networks
- B2
- B0
- B0 [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badg

Facetorch

Install / Use

README