<div align="center">

PETALface : Parameter Efficient Transfer Learning <br> for Low-resolution Face Recognition

<h3><strong>WACV 2025 (Oral)</strong></h3>

Kartik Narayan¹   Nithin Gopalakrishnan Nair¹   Jennifer Xu²   Rama Chellappa¹   Vishal M. Patel¹

Johns Hopkins University¹   Systems and Technology Research²

<a href='https://kartik-3004.github.io/PETALface/'><img src='https://img.shields.io/badge/Project-Page-blue'></a> <a href='https://arxiv.org/abs/2412.07771'><img src='https://img.shields.io/badge/Paper-arXiv-red'></a> <a href='https://huggingface.co/kartiknarayan/PETALface'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-orange'></a>

</div> <hr />

Motivation and Contribution

<p align="center" width="100%"> <img src='docs/static/images/intro_figure.png' height="75%" width="75%"> </p> Figure 1. (a) An illustration of the gallery and probe images from low-resolution dataset (BRIAR). Gallery images usually are high quality compared to the probe images. (b) t-SNE plot for the gallery and probe images of the BRIAR dataset. (c) Average CNN-IQA scores of gallery and probe images for 50 identities of the BRIAR dataset. <p align="center" width="100%"> <img src='docs/static/images/visual_abstract.png' height="75%" width="75%"> </p> Figure 2. The proposed PETAL<i>face</i>: a parameter efficient transfer learning approach adapts to low-resolution datasets beating the performance of pre-trained models with negligible drop in performance on high-resolution and mixed-quality datasets. PETAL<i>face</i> enables development of generalized models achieving competitive performance on high-resolution (LFW, CFP-FP, CPLFW, AgeDB, CALFW, CFP-FF) and mixed-quality datasets (IJB-B, IJB-C) with big enhancements in low-quality datasets (TinyFace, BRIAR, IJB-S). <br><br>

Low-resolution datasets contain images with poor clarity, making it challenging to extract meaningful discriminative features essential for face recognition and verification. Moreover, low-resolution datasets are usually small, with a limited number of subjects, as curating them requires significant time, effort, and investment. Existing methods force the learning of high-resolution and low-resolution face images in a single encoder, failing to account for the domain differences between them. From Figure 1(a), 1(b), and 1(c), we observe that high-quality gallery images and low-quality probe images belong to distinct domains, and require separate encoders to extract meaningful features for classification. A naive approach to adapting pre-trained models to low-resolution datasets is supervised full fine-tuning on these datasets. However, as mentioned, low-resolution datasets are small in size, and updating a model with a large number of parameters on a small low-resolution dataset results in poor convergence. This makes the model prone to catastrophic forgetting and we see a drop in performance on high-resolution and mixed-quality datasets, as shown in Figure 2.

With the above motivation,<br> 1️⃣ We introduce the use of the LoRA-based PETL technique to adapt large pre-trained face-recognition models to low-resolution datasets.<br> 2️⃣ We propose an image-quality-based weighting of LoRA modules to create separate proxy encoders for high-resolution and low-resolution data, ensuring effective extraction of embeddings for face recognition.<br> 3️⃣ We demonstrate the superiority of PETAL<i>face</i> in adapting to low-resolution datasets, outperforming other state-of-the-art models on low-resolution benchmarks while maintaining performance on high-resolution and mixed-quality datasets.<br>

<p align="justify"> Abstract: Pre-training on large-scale datasets and utilizing margin-based loss functions have been highly successful in training models for high-resolution face recognition. However, these models struggle with low-resolution face datasets, in which the faces lack the facial attributes necessary for distinguishing different faces. Full fine-tuning on low-resolution datasets, a naive method for adapting the model, yields inferior performance due to catastrophic forgetting of pre-trained knowledge. Additionally the domain difference between high-resolution (HR) gallery images and low-resolution (LR) probe images in low resolution datasets leads to poor convergence for a single model to adapt to both gallery and probe after fine-tuning. To this end, we propose PETALface, a Parameter-Efficient Transfer Learning approach for low-resolution face recognition. Through PETALface, we attempt to solve both the aforementioned problems. (1) We solve catastrophic forgetting by leveraging the power of parameter efficient fine-tuning(PEFT). (2) We introduce two low-rank adaptation modules to the backbone, with weights adjusted based on the input image quality to account for the difference in quality for the gallery and probe images. To the best of our knowledge, PETALface is the first work leveraging the powers of PEFT for low resolution face recognition. Extensive experiments demonstrate that the proposed method outperforms full fine-tuning on low-resolution datasets while preserving performance on high-resolution and mixed-quality datasets, all while using only 0.48% of the parameters. </p>

Framework

<p align="center" width="100%"> <img src='docs/static/images/petalface.png'> </p> Overview of the proposed PETAL<i>face</i> approach: We include an additional trainable module in linear layers present in attention layers and the final feature projection MLP. The trainable module is highlighted on the right. Specifically, we add two LoRA layers, where the weightage α is decided based on the input-image quality, computed using an off-the-shelf image quality assessment network (IQA).

:rocket: News

• [12/09/2025] 🔥 We release PETALface.

Installation

conda env create --file environment.yml
conda activate petalface

Download Data

The datasets can be downloaded from their respective webpages or by mailing the authors:<br>

1. WebFace4M, WebFace12M<br>
2. TinyFace<br>
3. BRIAR<br>
4. IJB-S<br>
5. IJB-B, IJB-C<br>
6. LFW<br>
7. CFP-FP<br>
8. CPLFW<br>
9. AgeDB-30<br>
10. CALFW<br>
11. CFP-FF<br>

Arrange the dataset in the following manner:

data/
├── BRIAR/
│   ├── train_set_1/
│   │   ├── train.idx
│   │   ├── train.lst
│   │   └── train.rec
│   └── train_set_2/
│       ├── train.idx
│       ├── train.lst
│       └── train.rec
├── HQ_val/
│   ├── agedb_30.bin
│   ├── calfw.bin
│   ├── cfp_ff.bin
│   ├── cfp_fp.bin
│   ├── cplfw.bin
│   └── lfw.bin
├── ijb/
│   ├── IJB_11.py
│   ├── IJBB/
│   ├── IJBC/
│   ├── recognition/
│   └── run.sh
├── IJBS/
│   ├── ijbs_participant_publication_consent.csv
│   ├── img/
│   ├── img.md5
│   ├── protocols/
│   ├── README.pdf
│   ├── videos/
│   ├── videos.md5
│   ├── videos_ground_truth/
│   └── videos_ground_truth.md5
├── tinyface_aligned_112/
│   ├── Gallery_Distractor/
│   ├── Gallery_Match/
│   ├── Probe/
│   ├── train.idx
│   ├── train.lst
│   └── train.rec
├── WebFace12M/
│   ├── train.idx
│   ├── train.lst
│   └── train.rec
└── WebFace4M/
    ├── train.idx
    ├── train.lst
    └── train.rec

Download Model weights

The pre-traind model can be downloaded manually from HuggingFace or using python:

from huggingface_hub import hf_hub_download

# Finetuned Weights

# The filename "swin_arcface_webface4m_tinyface" indicates that the model has a swin bakcbone and pretraind
# on webface4m dataset with arcface loss function and finetuned on tinyface.
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_arcface_webface4m_tinyface/model.pt", local_dir="./weights")
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_cosface_webface4m_tinyface/model.pt", local_dir="./weights")
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_cosface_webface4m_briar/model.pt", local_dir="./weights")
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_cosface_webface12m_briar/model.pt", local_dir="./weights")

# Pre-trained Weights
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_arcface_webface4m/model.pt", local_dir="./weights")
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_cosface_webface4m/model.pt", local_dir="./weights")
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_arcface_webface12m/model.pt", local_dir="./weights")
hf_hub_download(repo_id="kartiknarayan/PETALface", filename="swin_cosface_webface12m/model.pt", local_dir="./weights")

Usage

Download the trained