VideoProcessMining

The VideoProcessMining project provides a general approach to extract process-mining-conform event logs from unstructured video data. It targets custom datasets and spatio-temporal activity recognition.

It is an implementation of the reference architecture presented in "Shedding light on blind spots – Developing a reference architecture to leverage video data for process mining".

• Get latest version on ScienceDirect (works until June 17,2022)
• See article in Decision Support Systems
• arxiv

Dependencies

The VideoProcessMining project uses several open source computer vision projects.

• Detectron2 (License): For object detection
• Deep Sort with PyTorch (License): For object tracking
• PySlowFast (License): For activity recognition
• PM4Py (License): For XES event log generation
• Crepe Dataset from the paper STARE: Spatio-Temporal Attention Relocation for Multiple Structured Activities Detection
• imutils (License) for fast video frame reading

License

The VideoProcessMining project is released under the GNU General Public License v3.0.

We stated our changes in the modified files and included the original licenses in the licenses directory.

Functionality

The VideoProcessMining project comprises a GUI that automates and facilitates several tasks:

• Preprocessing of custom video datasets for spatio-temporal activity recognition
• Training and testing of activity recognition models
• Extraction of process-mining-conform event logs from unstructured video data (demo)

Installation

Please find installation instructions in INSTALL.md.

Dataset Preparation

You may follow the instructions in DATASET.md to prepare your custom dataset.

Use VideoProcessMining For Training Or Testing Models

After preparing your dataset, you can follow the instructions in INSTRUCTIONS.md to train and test your models.

You can also extract information from video data.

Reproduce our results

If you are interested in reproducing our results, follow the instructions in CREPE_DEMO.md.