Introduction

MTF is a highly efficient and extensible library for registration based tracking, also called direct visual tracking. It utilizes a modular decomposition of trackers in this domain wherein each tracker comprises the following 3 modules:

Search Method (SM): ESM, IC, IA, FC, FA, LM, NN, PF, RANSAC or LMS
Appearance Model (AM): SSD, SAD, ZNCC, SCV, LSCV, NCC, MI, CCRE, KLD, SSIM, SPSS, RIU, NGF, PCA or DFM
- Illumination Model (ILM): GB, PGB and RBF
State Space Model (SSM): Spline (50+ DOF), TPS (50+ DOF), Homography (8 DOF), Affine (6 DOF), ASRT (Anisotropic Scaling, Rotation and Translation - 5 DOF ), Similitude (4 DOF), AST (Anisotropic Scaling and Translation - 4 DOF), Isometry (3 DOF), IST (Isotropic Scaling and Translation - 3 DOF ) or Translation (2 DOF)

Documentation

Please refer the following documents for more details:

msc thesis [pdf]
system design papers
- IROS 2017 [pdf]
- CVIU (draft) [pdf]
results papers

There is also a dedicated website where Doxygen documentation will soon be available along with detailed tutorials and examples. It also provides several datasets formatted to work with MTF.

Implementation

The library is implemented entirely in C++ though interfaces for Python and MATLAB are also provided to aid its use in research applications. A simple interface for ROS is likewise provided for seamless integration with robotics projects. In addition to the registration tracking modules, MTF comes bundled with several state of the art learning and detection based trackers whose C++ implementations are publicly available - DSST, KCF, CMT, TLD, RCT, MIL, Struck, FragTrack, GOTURN and DFT. It can thus be used as a general purpose tracking test bed too.

Platforms

MTF supports both Unix and Windows platforms. Though the Unix build system has been tested comprehensively only under Linux, specifically Ubuntu 14.04/16.04, it should work on Macintosh systems too (see Installation Issues section below for resolving possible issues). The Windows build system has been tested on Windows 8.1 and 10 with Visual Studio 2015, 2017 and 2019 though it should work fine with any non-ancient versions of the OS and IDE.

License and citing

MTF is provided under BSD license and so is free for research and commercial applications. We do request, however, that this paper [bibtex] [arxiv version] be cited by any publications resulting from projects that use MTF so more people can get to know about and benefit from it. Finally, if any issues are encountered while installing or running the library, please create an entry in the issues section and we will do our best to resolve it as soon as possible.

Prerequisites

General

Required

compiler supporting C++11 (GCC 4.7 or newer)
Eigen
- needs to be added to the C/C++ include paths which can be done, for instance (linux), by running the following commands, assuming that Eigen is installed in /usr/include/eigen3:
  echo "export C_INCLUDE_PATH=${C_INCLUDE_PATH}:/usr/include/eigen3" >> ~/.bashrc
  echo "export CPLUS_INCLUDE_PATH=${CPLUS_INCLUDE_PATH}:/usr/include/eigen3" >> ~/.bashrc
- version 3.2.10 or newer is needed
OpenCV
- comprehensive testing has only been done with versions 2.4.13, 3.3.0 and 3.4.1;
- 2.4.x is recommended for widest compatibility as rare issues with OpenCV 3.x have been known to prevent successful compilation with some configurations (see Installation Issues section below)
- 3.3.x or 3.4.x is recommended for widest functionality
- 4.x.x is *

MTF

Install / Use

README