BillySTAT

BillySTAT records your Snooker statistics using ~~YOLOv3~~, OpenCV ~~and NVidia Cuda~~.

<a href="https://i.imgur.com/hZDxVAB.png"><img src="https://i.imgur.com/hZDxVAB.png" title="HUD for BillySTAT" /></a>

<a href="https://i.imgur.com/fDvP2RI.png"><img src="https://i.imgur.com/fDvP2RI.png" title="GUI for BillySTAT" /></a>

How to use BillySTAT:

In folder billystat/kristian-kesken/gui/ run command

python billystatApp.py

Choose video to analyse: File->Open and choose the video and press Start game

Next

Click outermost points of the snooker table on the displayed video for mask and press C. Then mark the pockets by clicking on them and press C again. It should start running.

Simple as that.

Our achievements in action @ https://www.youtube.com/channel/UCiSQy3Upsj8ocebaWwPHvFQ

<!-- toc -->

• BillySTAT
  • School project @Haaga-Helia University of Applied Sciences
    • Project members
    • Things to do
  • Information page
  • Setting up environment for YOLOv3
    • Setting up server
      • Creating access point for remote work
  • Installations
    • Nvidia drivers
    • CUDA installation
    • OpenCV3
    • Installing Darknet
    • Installing Nextcloud for cloud-storage with docker-compose
    • Nextcloud client for easy syncing
  • Training your neural networks
    • Creating training material
      • Resizing images
      • 1st Alternative: YOLO-Annotation-Tool
      • 2nd Alternative: Open Labeling
      • 3rd Alternative: Yolo_mark by AlexeyAB
  • Actual training
    • Marking the images
  • YOLOv3 Does not suit for detecting fast small objects
  • Testing frame difference from video
    • Virtualenv
    • Frame diff from video with grayscale
    • Frame diff with color
    • Getting more material
  • OpenCV Object selection by color, cv2.HoughCircles
    • Defining ROIs
    • Working on the Pallo.py
  • Possible solutions to detecting balls from an angled view
    • Subtracting/Comparing frames
      • Empty game area without Snooker-balls using GIMP and G'MIC
    • 2D Perspective warping
    • Solution to our recognition problem (Birdview footage)
  • Creating a GUI for BillySTAT
    • Tkinter
    • Mockup
    • Creating the GUI
      • Dropdown menu
      • Pushable buttons
      • Final state of GUI
    • Displaying statistics on screen
    • Saving the statistics
  • GoPro as the camera
  • Summary
    • YoloV3
    • OpenCV3
    • Thoughts
  • References

<small><i><a href='http://ecotrust-canada.github.io/markdown-toc/'>Table of contents generated with markdown-toc</a></i></small>

<!-- tocstop -->

BillySTAT

BillySTAT records your Snooker statistics using ~~YOLOv3~~, OpenCV ~~and NVidia Cuda~~.

• Darknet
• YOLOv3
• OpenCV3

School project @Haaga-Helia University of Applied Sciences

Project members

• Kristian Syrjänen
• Axel Rusanen
• Miikka Valtonen Project Manager
• Matias Richterich

Done on Xubuntu 18.04 LTS

Done with

• Darknet
• YOLOv3
• OpenCV3

Hardware HP z820, Xeon e52630 v2 x2, 2 x 8gb 1333 mHz per per processor, 1 TB SSHD

Images of our setup

Things to do

PRIO 1

• ~~Train new YOLOv3 weight!~~
• ~~Make OpenCV detection work!~~
• ~~Setup server~~
• ~~Install CUDA, OpenCV3, Darknet and YOLOv3~~
• ~~Run basic tests~~
• ~~Make it recognise only things relevant~~
• Create boundaries that when crossed, counts as a point/points depending what colored ball it is.[Undertesting]
• ~~Make it count statistics~~
• ~~Create GUI for statistics~~
• ~~Gather image material~~

Information page

Thoughts, Ideas and Problems

Setting up environment for YOLOv3

(Laptop users attention: Getting your discrete gpu to work will be a driver-nightmare)

Setting up server

Creating access point for remote work

Config changes on 01-network-manager-all.yaml

network:
ethernets:
   enp1s0:
     addresses: [192.168.1.2/24]
     gateway4: 192.168.1.1
     nameservers:
       addresses: [1.1.1.1,8.8.8.8]
     dhcp4: no
 version: 2

And on our router we enabled port-forwarding to a desired port.

Installations

Nvidia drivers

First off we'll download NVidia drivers, let's start by adding nvidia ppa:latest,

sudo add-apt-repository ppa:graphics-drivers
sudo apt-get update

Install Nvidia drivers, (NOTE! At the time of writing, Cuda 10 FORCES 410 drivers. Meaning, if you have 415 or newer drivers installed, they will be uninstalled and replaced with 410 drivers. With some work this can be avoided, but you can just reinstall the newer drivers afterwards if necessary.)

sudo apt-get install nvidia-driver-410

And reboot

sudo reboot

CUDA installation

Head on to the download page, download the needed file and proceed with instructions.

sudo dpkg -i cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
sudo apt-get update
sudo apt-get install cuda

We had trouble with apt-get so we used aptitude.

sudo apt-get install aptitude
sudo aptitude install cuda

Reboot and try out nvidia-smi

nvidia-smi

OpenCV3

This is taken from the OpenCV3 installation page.

Install OS libraries

sudo apt-get update
sudo apt-get upgrade

sudo apt-get remove x264 libx264-dev

sudo apt-get install build-essential checkinstall cmake pkg-config yasm
sudo apt-get install git gfortran
sudo apt-get install libjpeg8-dev libjasper-dev libpng12-dev

sudo apt-get install libtiff5-dev

sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev
sudo apt-get install libxine2-dev libv4l-dev
sudo apt-get install libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev
sudo apt-get install qt5-default libgtk2.0-dev libtbb-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install libfaac-dev libmp3lame-dev libtheora-dev
sudo apt-get install libvorbis-dev libxvidcore-dev
sudo apt-get install libopencore-amrnb-dev libopencore-amrwb-dev
sudo apt-get install x264 v4l-utils

sudo apt-get install libprotobuf-dev protobuf-compiler
sudo apt-get install libgoogle-glog-dev libgflags-dev
sudo apt-get install libgphoto2-dev libeigen3-dev libhdf5-dev doxygen

sudo apt-get install python-dev python-pip python3-dev python3-pip
sudo -H pip2 install -U pip numpy
sudo -H pip3 install -U pip numpy

Install Python libraries

sudo pip2 install virtualenv virtualenvwrapper
sudo pip3 install virtualenv virtualenvwrapper
echo "# Virtual Environment Wrapper"  >> ~/.bashrc
echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.bashrc
source ~/.bashrc

mkvirtualenv facecourse-py3 -p python3
workon facecourse-py3

pip install numpy scipy matplotlib scikit-image scikit-learn ipython
// Exit virtual environment with deactivate
deactivate

Download opencv from Github

git clone https://github.com/opencv/opencv.git
cd opencv 
git checkout 3.3.1 
cd ..

Download opencv_contrib from Github

git clone https://github.com/opencv/opencv_contrib.git
cd opencv_contrib
git checkout 3.3.1
cd ..

Compile and install OpenCV with contrib modules Create a build directory

cd opencv
mkdir build
cd build

Run CMake

cmake -D CMAKE_BUILD_TYPE=RELEASE \
  -D CMAKE_INSTALL_PREFIX=/usr/local \
  -D INSTALL_C_EXAMPLES=OFF \
  -D INSTALL_PYTHON_EXAMPLES=OFF \
  -D WITH_TBB=ON \
  -D WITH_V4L=ON \
  -D WITH_QT=ON \
  -D WITH_OPENGL=ON \
  -D WITH_GSTREAMER=ON \
  -D WITH_CUDA=ON \
  -D WITH_NVCUVID=ON \
  -D ENABLE_FAST_MATH=1 \
  -D CUDA_F