StereoMatchingTFG

Contents

1. Introduction
2. Usage
3. Results

Introduction

TO DO

Usage

Setup

Install conda and import virtual environment from stereo.yml with:

conda env create -f stereo.yml

Activate the virtual environment with:

conda activate stereo

Download the following datasets:

• KITTI Stereo 2015
• Scene Flow

Usage of KITTI 2015 dataset:
Download stereo 2015/flow 2015/scene flow 2015 data set (2 GB)

Usage of Scene Flow dataset:
Download RGB finalpass images and its disparity for three subset: FlyingThings3D, Driving, and Monkaa.
Create the folders frames_finalpass, disparity and camera_data. In each one create the folders TRAIN and TEST.
Locate Driving and Monkaa intern folders in TRAIN and FlyingThings3D has TRAIN and TEST folders.

Change the datasets paths in dataloaders/__init__.py

Train

To train on Sceneflow use the following command:

sh train.sh

That contains the following code:

CUDA_VISIBLE_DEVICES=0 python train.py \
                       --dataset 'sceneflow' \
                       --batchsize 32 \
                       --total_epochs 40 \
                       --start_epoch 0 \
                       --workers_train 8 \
                       --workers_test 8 \
                       --learnrate  0.0001 \
                       --model DispNet \
                       --logdir 'tensorboard/finetune' \
                       --pth_name 'DispNetSceneFlow' \
                       --maxdisp 192

Tensorboard will save the loss, metrics and outputs on tensorboard/train.

The pth will be saved on /Vitis/build/float_model/DispNetSceneFlow.pth.

The old format pth will be saved on /Vitis/build/float_model/DispNetSceneFlow_old.pth.

Finetune

To finetune on KITTI 2015 use the following command:

sh finetune.sh

That contains the following code:

CUDA_VISIBLE_DEVICES=0 python train.py \
                       --dataset 'kitti' \
                       --batchsize 32 \
                       --total_epochs 300 \
                       --start_epoch 0 \
                       --workers_train 8 \
                       --workers_test 8 \
                       --learnrate 0.0001 \
                       --model DispNet \
                       --logdir 'tensorboard/finetune' \
                       --pth_name 'DispNetKITTI.pth' \
                       --maxdisp 192 \
                       --load_model './Vitis/build/float_model/DispNetSceneFlow.pth'

Tensorboard will save the loss, metrics and outputs on tensorboard/finetune.

The pth will be saved on /Vitis/build/float_model/DispNetKITTI.pth.

The old format pth will be saved on /Vitis/build/float_model/DispNetKITTI_old.pth.

Prediction

TO DO

Pretrained Model

TO DO

Tensorboard

To check loss, metrics and outputs of execution use the following command in tensorboard directory:

tensorboard --logdir (train or finetune) --port (desired port)

If you are connecting via ssh the following option can be added to the ssh command to redirect the remote port to your local machine:

ssh ... -L (local port):127.0.0.1:(remote port) ...

<<<<<<< HEAD

Creación xmodel de la red para ejecutar con Alveo U50 (Vitis Ai)

=======

Creación xmodel de la red para ejecutar con Alveo U50 (Vitis Ai)

0c2cb66ea0d2022ff30064336d324de32adc6351 Se genera el modelo cuantizado en el directorio ./Vitis/build/quant_model y se compila para poder ejecutar en Alveo U50

./docker_run.sh xilinx/vitis-ai-cpu:latest
pip install scikit-image
conda activate vitis-ai-pytorch
cd Vitis
sh quantize.sh
source compile.sh u50
sh target.sh

Ejecutar en Alveo U50

(TO DO)

Results

40 images

Rendimiento RTX 2080 Ti -> Batchsize 7 -> 1.8891s - 21.17 fps RTX 3090 -> Batchsize 10 -> 1.9077s - 20.97 fps Alveo U50 -> Batchsize 3(forced) -> 4.3024s - 9.3 fps CPU -> Batchsize 10 -> 19.7868s - 2.02 fps

Precision DispNet Mejorada Versión Pytorch = 3.539% pixeles erroneos de media Versión Vitis = 39.096% pixeles erroneos de media

DispNet Versión Pytorch = 6.3305% pixeles erroneos de media

This repo bring from StereoNet and PSMNet.