Nni

<div align="center"> <img src="docs/img/nni_logo.png" width="600"/> </div> <br/>

<img src="docs/img/readme_banner.png" width="100%"/>

NNI automates feature engineering, neural architecture search, hyperparameter tuning, and model compression for deep learning. Find the latest features, API, examples and tutorials in our official documentation (简体中文版点这里).

What's NEW!  <a href="#nni-released-reminder"><img width="48" src="docs/img/release_icon.png"></a>

• New release: v3.0 preview is available - released on May-5-2022
• New demo available: Youtube entry | Bilibili 入口 - last updated on June-22-2022
• New research paper: SparTA: Deep-Learning Model Sparsity via Tensor-with-Sparsity-Attribute - published in OSDI 2022
• New research paper: Privacy-preserving Online AutoML for Domain-Specific Face Detection - published in CVPR 2022
• Newly upgraded documentation: Doc upgraded

Installation

See the NNI installation guide to install from pip, or build from source.

To install the current release:

$ pip install nni

To update NNI to the latest version, add --upgrade flag to the above commands.

NNI capabilities in a glance

<img src="docs/img/overview.svg" width="100%"/> <table> <tbody> <tr align="center" valign="bottom"> <td></td> <td> <b>Hyperparameter Tuning</b> <img src="docs/img/bar.png" /> </td> <td> <b>Neural Architecture Search</b> <img src="docs/img/bar.png" /> </td> <td> <b>Model Compression</b> <img src="docs/img/bar.png" /> </td> </tr> <tr valign="top"> <td align="center" valign="middle"> <b>Algorithms</b> </td> <td> <ul> <li><b>Exhaustive search</b></li> <ul> <li><a href="https://nni.readthedocs.io/en/latest/reference/hpo.html#nni.algorithms.hpo.gridsearch_tuner.GridSearchTuner">Grid Search</a></li> <li><a href="https://nni.readthedocs.io/en/latest/reference/hpo.html#nni.algorithms.hpo.random_tuner.RandomTuner">Random</a></li> </ul> <li><b>Heuristic search</b></li> <ul> <li><a href="https://nni.readthedocs.io/en/latest/reference/hpo.html#nni.algorithms.hpo.hyperopt_tuner.HyperoptTuner">Anneal</a></li> <li><a href="https://nni.readthedocs.io/en/latest/reference/hpo.html#nni.algorithms.hpo.evolution_tuner.EvolutionTuner">Evolution</a></li> <li><a href="https://nni.readthedocs.io/en/latest/reference/hpo.html#nni.algorithms.hpo.hyperband_advisor.Hyperband">Hyperband</a></li> <li><a href="https://nni.readthedocs.io/en/latest/reference/hpo.html#nni.algorithms.hpo.pbt_tuner.PBTTuner">PBT</a></li> </ul> <li><b>Bayesian optimization</b></li> <ul> <li><a href="https://nni.readthedocs.io/en/latest/reference/hpo.html#nni.algorithms.hpo.bohb_advisor.BOHB">BOHB</a></li> <li><a href="https://nni.readthedocs.io/en/latest/reference/hpo.html#nni.algorithms.hpo.dngo_tuner.DNGOTuner">DNGO</a></li> <li><a href="https://nni.readthedocs.io/en/latest/reference/hpo.html#nni.algorithms.hpo.gp_tuner.GPTuner">GP</a></li> <li><a href="https://nni.readthedocs.io/en/latest/reference/hpo.html#nni.algorithms.hpo.metis_tuner.MetisTuner">Metis</a></li> <li><a href="https://nni.readthedocs.io/en/latest/reference/hpo.html#nni.algorithms.hpo.smac_tuner.SMACTuner">SMAC</a></li> <li><a href="https://nni.readthedocs.io/en/latest/reference/hpo.html#nni.algorithms.hpo.tpe_tuner.TpeTuner">TPE</a></li> </ul> </ul> </td> <td> <ul> <li><b>Multi-trial</b></li> <ul> <li><a href="https://nni.readthedocs.io/en/latest/nas/exploration_strategy.html#grid-search-strategy">Grid Search</a></li> <li><a href="https://nni.readthedocs.io/en/latest/nas/exploration_strategy.html#policy-based-rl-strategy">Policy Based RL</a></li> <li><a href="https://nni.readthedocs.io/en/latest/nas/exploration_strategy.html#random-strategy">Random</a></li> <li><a href="https://nni.readthedocs.io/en/latest/nas/exploration_strategy.html#regularized-evolution-strategy">Regularized Evolution</a></li> <li><a href="https://nni.readthedocs.io/en/latest/nas/exploration_strategy.html#tpe-strategy">TPE</a></li> </ul> <li><b>One-shot</b></li> <ul> <li><a href="https://nni.readthedocs.io/en/latest/nas/exploration_strategy.html#darts-strategy">DARTS</a></li> <li><a href="https://nni.readthedocs.io/en/latest/nas/exploration_strategy.html#enas-strategy">ENAS</a></li> <li><a href="https://nni.readthedocs.io/en/latest/nas/exploration_strategy.html#fbnet-strategy">FBNet</a></li> <li><a href="https://nni.readthedocs.io/en/latest/nas/exploration_strategy.html#proxylessnas-strategy">ProxylessNAS</a></li> <li><a href="https://nni.readthedocs.io/en/latest/nas/exploration_strategy.html#spos-strategy">SPOS</a></li> </ul> </ul> </td> <td> <ul> <li><b>Pruning</b></li> <ul> <li><a href="https://nni.readthedocs.io/en/latest/compression/pruner.html#level-pruner">Level</a></li> <li><a href="https://nni.readthedocs.io/en/latest/compression/pruner.html#l1-norm-pruner">L1 Norm</a></li> <li><a href="https://nni.readthedocs.io/en/latest/compression/pruner.html#taylor-fo-weight-pruner">Taylor FO Weight</a></li> <li><a href="https://nni.readthedocs.io/en/latest/compression/pruner.html#movement-pruner">Movement</a></li> <li><a href="https://nni.readthedocs.io/en/latest/compression/pruner.html#agp-pruner">AGP</a></li> <li><a href="https://nni.readthedocs.io/en/latest/compression/pruner.html#auto-compress-pruner">Auto Compress</a></li> <li><a href="https://nni.readthedocs.io/en/latest/compression/pruner.html">More...</a></li> </ul> <li><b>Quantization</b></li> <ul> <li><a href="https://nni.readthedocs.io/en/latest/compression/quantizer.html#naive-quantizer">Naive</a></li> <li><a href="https://nni.readthedocs.io/en/latest/compression/quantizer.html#qat-quantizer">QAT</a></li> <li><a href="https://nni.readthedocs.io/en/latest/compression/quantizer.html#lsq-quantizer">LSQ</a></li> <li><a href="https://nni.readthedocs.io/en/latest/compression/quantizer.html#observer-quantizer">Observer</a></li> <li><a href="https://nni.readthedocs.io/en/latest/compression/quantizer.html#dorefa-quantizer">DoReFa</a></li> <li><a href="https://nni.readthedocs.io/en/latest/compression/quantizer.html#bnn-quantizer">BNN</a></li> </ul> </ul> </td> <tr align="center" valign="bottom"> <td></td> <td> <b>Supported Frameworks</b> <img src="docs/img/bar.png" /> </td> <td> <b>Training Services</b> <img src="docs/img/bar.png" /> </td> <td> <b>Tutorials</b> <img src="docs/img/bar.png" /> </td> </tr> <tr valign="top"> <td align="center" valign="middle"> <b>Supports</b> </td> <td> <ul> <li>PyTorch</li> <li>TensorFlow</li> <li>Scikit-learn</li> <li>XGBoost</li> <li>LightGBM</li> <li>MXNet</li> <li>Caffe2</li> <li>More...</li> </ul> </td> <td> <ul> <li><a href="https://nni.readthedocs.io/en/latest/experiment/local.html">Local machine</a></li> <li><a href="https://nni.readthedocs.io/en/latest/experiment/remote.html">Remote SSH servers</a></li> <li><a href="https://nni.readthedocs.io/en/latest/experiment/aml.html">Azure Machine Learning (AML)</a></li> <li><b>Kubernetes Based</b></li> <ul> <li><a href="https://nni.readthedocs.io/en/latest/experiment/openpai.html">OpenAPI</a></li> <li><a href="https://nni.readthedocs.io/en/latest/experiment/kubeflow.html">Kubeflow</a></li> <li><a href="https://nni.readthedocs.io/en/latest/experiment/frameworkcontroller.html">FrameworkController</a></li> <li><a href="https://nni.readthedocs.io/en/latest/experiment/adaptdl.html">AdaptDL</a></li> <li><a href="https://nni.readthedocs.io/en/latest/experiment/paidlc.html">PAI DLC</a></li> </ul> <li><a href="https://nni.readthedocs.io/en/latest/experiment/hybrid.html">Hybrid training services</a></li> </ul> </td> <td> <ul> <li><b>HPO</b></li> <ul> <li><a href="https://nni.readthedocs.io/en/latest/tutorials/hpo_quickstart_pytorch/main.html">PyTorch</a></li> <li><a href="https://nni.readthedocs.io/en/latest/tutorials/hpo_quickstart_tensorflow/main.html">TensorFlow</a></li> </ul> <li><b>NAS</b></li> <ul> <li><a href="https://nni.readthedocs.io/en/latest/tutorials/hello_nas.html">Hello NAS</a></li> <li><a href="https://nni.readthedocs.io/en/latest/tutorials/nasbench_as_dataset.html">NAS Benchmarks</a></li> </ul> <li><b>Compression</b></li> <ul> <li><a href="https://nni.readthedocs.io/en/latest/tutorials/pruning_quick_start.html">Pruning</a></li> <li><a href="https://nni.readthedocs.io/en/latest/tutorials/pruning_speed_up.html">Pruning Speedup</a></li> <li><a href="https://nni.readthedocs.io/en/latest/tutorials/quantization_quick_start.html">Quantization</a></li> <li><a href="https://nni.readthedocs.io/en/latest/tutorials/quantization_speed_up.html">Quantization Speedup</a></li> </ul> </ul> </td>