LCCNet

Official PyTorch implementation of the paper “LCCNet: Lidar and Camera Self-Calibration Using Cost Volume Network”. A video of the demonstration of the method can be found on https://www.youtube.com/watch?v=UAAGjYT708A

Table of Contents

• Requirements
• Pre-trained model
• Evaluation
• Train
• Citation

Requirements

• python 3.6 (recommend to use Anaconda)
• PyTorch==1.0.1.post2
• Torchvision==0.2.2
• Install requirements and dependencies

pip install -r requirements.txt

Pre-trained model

Pre-trained models can be downloaded from google drive

Evaluation

1. Download KITTI odometry dataset.
2. Change the path to the dataset in evaluate_calib.py.

data_folder = '/path/to/the/KITTI/odometry_color/'

3. Create a folder named pretrained to store the pre-trained models in the root path.
4. Download pre-trained models and modify the weights path in evaluate_calib.py.

weights = [
   './pretrained/kitti_iter1.tar',
   './pretrained/kitti_iter2.tar',
   './pretrained/kitti_iter3.tar',
   './pretrained/kitti_iter4.tar',
   './pretrained/kitti_iter5.tar',
]

5. Run evaluation.

python evaluate_calib.py

Train

python train_with_sacred.py

Citation

Thank you for citing our paper if you use any of this code or datasets.

@article{lv2020lidar,
  title={Lidar and Camera Self-Calibration using CostVolume Network},
  author={Lv, Xudong and Wang, Boya and Ye, Dong and Wang, Shuo},
  journal={arXiv preprint arXiv:2012.13901},
  year={2020}
}

Acknowledgments

We are grateful to Daniele Cattaneo for his CMRNet github repository. We use it as our initial code base.

<!-- [correlation_package](models/LCCNet/correlation_package) was taken from [flownet2](https://github.com/NVIDIA/flownet2-pytorch/tree/master/networks/correlation_package) [LCCNet.py](model/LCCNet.py) is a modified version of the original [PWC-DC network](https://github.com/NVlabs/PWC-Net/blob/master/PyTorch/models/PWCNet.py) and modified version [CMRNet](https://github.com/cattaneod/CMRNet/blob/master/models/CMRNet/CMRNet.py) -->

---