Efficient Video Compression via Content-Adaptive Super-Resolution

We present a new approach that augments existing codecs with a small, content-adaptive super-resolution model that significantly boosts video quality. Our method, SRVC, encodes video into two bitstreams:

1. a content stream, produced by compressing downsampled low-resolution video with the existing codec,
2. a model stream, which encodes periodic updates to a lightweight super-resolution neural network customized for short segments of the video.

SRVC decodes the video by passing the decompressed low-resolution video frames through the (time-varying) super-resolution model to reconstruct high-resolution video frames.

Installation

For installing the required packages using Conda, use:

git clone https://github.com/AdaptiveVC/SRVC.git
cd SRVC
conda env create -f environment.yml

Running SRVC

For running SRVC, check python srvc.py --help.

Videos

Vimeo Short Films

We use 28 short films with direct download buttons on Vimeo. These videos are in high-resolution, have realistic scene changes from movie-makers, and have 10min+ duration.

Vimeo video_ids are located here, and you can view/download them at vimeo.com/video_id, e.g., vimeo.com/441417334.

Once downloaded, you can use the scripts and instructions located here to pre-process the data into the low-resolution and high-resolution formats needed by SRVC.

Xiph Full Sequences

We use the following four long video sequences from the Xiph video dataset:

• Big Buck Bunny
• Elephants Dream
• Sita Sings the Blues
• Meridian

Citation

You may cite this work using:

@InProceedings{Khani_2021_ICCV,
    author    = {Khani, Mehrdad and Sivaraman, Vibhaalakshmi and Alizadeh, Mohammad},
    title     = {Efficient Video Compression via Content-Adaptive Super-Resolution},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {4521-4530}
}