Flux Project

Autodeploy a complete end-to-end machine/deep learning pipeline on Kubernetes using tools like Spark, TensorFlow, HDFS, etc. - it requires a running Kubernetes (K8s) cluster in the cloud or on-premise.

Please visit the website for updates.

<img src="./images/flux_overview.png" width="418">

Prerequisites

Before installing the components make sure you have installed

• Docker The edge version of docker community edition is coming with a kubernetes option
• Kubernetes
• Helm The package manager for Kubernetes.

Deploy on nodes

./bin/flux will check for GPU availability and make use of it if it can find a GPU.

1. Build the images

   ./bin/flux build
   

   Note that images need to be deployed to your nodes or to your docker registry

2. Create the deployment and the service with Kubernetes

   ./bin/flux start
   

3. Check that all components are running

   ./bin/flux ps
   

Accessing the sample notebooks:

./bin/flux notebook

A browser window opens. You can there login using flux/flux.

<img src="./images/login_notebook.png" height="500" width="800">

After Login an ipython notebook playground with examples will open.

<img src="./images/sample_notebook.png" height="500" width="800">

Cloud deployment

<img src="./images/flux_cloud.png" width="348">