Clearml

<div align="center" style="text-align: center"> <p style="text-align: center"> <img align="center" src="docs/clearml-logo.svg#gh-light-mode-only" alt="Clear|ML"><img align="center" src="docs/clearml-logo-dark.svg#gh-dark-mode-only" alt="Clear|ML"> </p>

ClearML - Auto-Magical Suite of tools to streamline your AI workflow </br>Experiment Manager, MLOps/LLMOps and Data-Management

<br>

🌟 ClearML is open-source - Leave a star to support the project! 🌟

</div>

---

ClearML

ClearML is a ML/DL development and production suite. It contains FIVE main modules:

• Experiment Manager - Automagical experiment tracking, environments and results
• MLOps / LLMOps - Orchestration, Automation & Pipelines solution for ML/DL/GenAI jobs (Kubernetes / Cloud / bare-metal)
• Data-Management - Fully differentiable data management & version control solution on top of object-storage (S3 / GS / Azure / NAS)
• Model-Serving - cloud-ready Scalable model serving solution!
  • Deploy new model endpoints in under 5 minutes
  • Includes optimized GPU serving support backed by Nvidia-Triton
  • with out-of-the-box Model Monitoring
• Reports - Create and share rich MarkDown documents supporting embeddable online content
• :fire: Orchestration Dashboard - Live rich dashboard for your entire compute cluster (Cloud / Kubernetes / On-Prem)
• 🔥 💥 Fractional GPUs - Container based, driver level GPU memory limitation 🙀 !!!

Instrumenting these components is the ClearML-server, see Self-Hosting & Free tier Hosting

---

<div align="center">

Sign up & Start using in under 2 minutes

---

Friendly tutorials to get you started

<table> <tbody> <tr> <td><a href="https://github.com/clearml/clearml/blob/master/docs/tutorials/Getting_Started_1_Experiment_Management.ipynb"><b>Step 1</b></a> - Experiment Management</td> <td><a target="_blank" href="https://colab.research.google.com/github/clearml/clearml/blob/master/docs/tutorials/Getting_Started_1_Experiment_Management.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a></td> </tr> <tr> <td><a href="https://github.com/clearml/clearml/blob/master/docs/tutorials/Getting_Started_2_Setting_Up_Agent.ipynb"><b>Step 2</b></a> - Remote Execution Agent Setup</td> <td><a target="_blank" href="https://colab.research.google.com/github/clearml/clearml/blob/master/docs/tutorials/Getting_Started_2_Setting_Up_Agent.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a></td> </tr> <tr> <td><a href="https://github.com/clearml/clearml/blob/master/docs/tutorials/Getting_Started_3_Remote_Execution.ipynb"><b>Step 3</b></a> - Remotely Execute Tasks</td> <td><a target="_blank" href="https://colab.research.google.com/github/clearml/clearml/blob/master/docs/tutorials/Getting_Started_3_Remote_Execution.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a></td> </tr> </tbody> </table> </div>

---

<table> <tbody> <tr> <td>Experiment Management</td> <td>Datasets</td> </tr> <tr> <td><a href="https://app.clear.ml"><img src="https://github.com/clearml/clearml/blob/master/docs/experiment_manager.gif?raw=true" width="100%"></a></td> <td><a href="https://app.clear.ml/datasets"><img src="https://github.com/clearml/clearml/blob/master/docs/datasets.gif?raw=true" width="100%"></a></td> </tr> <tr> <td colspan="2" height="24px"></td> </tr> <tr> <td>Orchestration</td> <td>Pipelines</td> </tr> <tr> <td><a href="https://app.clear.ml/workers-and-queues/autoscalers"><img src="https://github.com/clearml/clearml/blob/master/docs/orchestration.gif?raw=true" width="100%"></a></td> <td><a href="https://app.clear.ml/pipelines"><img src="https://github.com/clearml/clearml/blob/master/docs/pipelines.gif?raw=true" width="100%"></a></td> </tr> </tbody> </table>

ClearML Experiment Manager

Adding only 2 lines to your code gets you the following

• Complete experiment setup log
  • Full source control info, including non-committed local changes
  • Execution environment (including specific packages & versions)
  • Hyper-parameters
    • argparse/Click/PythonFire for command line parameters with currently used values
    • Explicit parameters dictionary
    • Tensorflow Defines (absl-py)
    • Hydra configuration and overrides
  • Initial model weights file
• Full experiment output automatic capture
  • stdout and stderr
  • Resource Monitoring (CPU/GPU utilization, temperature, IO, network, etc.)
  • Model snapshots (With optional automatic upload to central storage: Shared folder, S3, GS, Azure, Http)
  • Artifacts log & store (Shared folder, S3, GS, Azure, Http)
  • Tensorboard/TensorboardX scalars, metrics, histograms, images, audio and video samples
  • Matplotlib & Seaborn
  • ClearML Logger interface for complete flexibility.
• Extensive platform support and integrations
  • Supported ML/DL frameworks: PyTorch (incl' ignite / lightning), Tensorflow, Keras, AutoKeras, FastAI, XGBoost, LightGBM, MegEngine and Scikit-Learn
  • Seamless integration (including version control) with Jupyter Notebook and PyCharm remote debugging

Start using ClearML

1. Sign up for free to the ClearML Hosted Service (alternatively, you can set up your own server, see here).

   ClearML Demo Server: ClearML no longer uses the demo server by default. To enable the demo server, set the CLEARML_NO_DEFAULT_SERVER=0 environment variable. Credentials aren't needed, but experiments launched to the demo server are public, so make sure not to launch sensitive experiments if using the demo server.

2. Install the clearml python package:

   pip install clearml
   

3. Connect the ClearML SDK to the server by creating credentials, then execute the command below and follow the instructions:

   clearml-init
   

4. Add two lines to your code:

   from clearml import Task
   task = Task.init(project_name='examples', task_name='hello world')
   

And you are done! Everything your process outputs is now automagically logged into ClearML.

Next step, automation! Learn more about ClearML's two-click automation here.

ClearML Architecture

The ClearML run-time components:

• The ClearML Python Package - for integrating ClearML into your existing scripts by adding just two lines of code, and optionally extending your experiments and other workflows