Iterative Residual Refinement <br/> for Joint Optical Flow and Occlusion Estimation

<img src=output.gif>

This repository is the PyTorch implementation of the paper:

Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation (CVPR 2019)
Junhwa Hur and Stefan Roth
Department of Computer Science, TU Darmstadt
[Preprint]   [Proceeding]   [Supplemental]

Please cite the paper below if you find our paper and source codes are useful.

@inproceedings{Hur:2019:IRR,  
  Author = {Junhwa Hur and Stefan Roth},  
  Booktitle = {CVPR},  
  Title = {Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation},  
  Year = {2019}  
}

Contact: junhwa.hur[at]visinf.tu-darmstadt.de

Getting started

This code has been orginally developed under Anaconda(Python 3.6), PyTorch 0.4.1 and CUDA 8.0 on Ubuntu 16.04.

1. Please install the followings:

   • Anaconda
   • PyTorch (now compatible with PyTorch 1.5.0)
   • tqdm (conda install -c conda-forge tqdm==4.40.0)
   • (any missing packages that the code requires)

2. The datasets used for this projects are followings:

   • FlyingChairsOcc dataset
   • FlyingThings3D subset
   • MPI Sintel Dataset + revised occlusion GT
   • KITTI Optical Flow 2015 and KITTI Optical Flow 2012

Training

The scripts folder contains training scripts of experiments demonstrated in the paper.
To train the model, you can simply run the script file, e.g., ./IRR-PWC_flyingChairsOcc.sh.
In script files, please configure your own experiment directory (EXPERIMENTS_HOME) and dataset directory in your local system (e.g., SINTEL_HOME or KITTI_HOME).

Pretrained Models

The saved_check_point contains the pretrained models of i) baseline, ii) baseline + irr, and iii) full models.
Additional pretrained models in the ablations study (Table 1 in the main paper) and their training scripts are available upon request.

Inference

The scripts for testing the pre-trained models are located in scripts/validation.

Acknowledgement

Portions of the source code (e.g., training pipeline, runtime, argument parser, and logger) are from Jochen Gast