<p align="center"> <a href="https://ludwig.ai"> <img src="https://github.com/ludwig-ai/ludwig-docs/raw/main/docs/images/ludwig_hero_smaller.jpg" height="150"> </a> </p> <div align="center">

Declarative deep learning framework built for scale and efficiency.

</div>

📖 What is Ludwig?

Ludwig is a low-code framework for building custom AI models like LLMs and other deep neural networks.

Key features:

• 🛠 Build custom models with ease: a declarative YAML configuration file is all you need to train a state-of-the-art LLM on your data. Support for multi-task and multi-modality learning. Comprehensive config validation detects invalid parameter combinations and prevents runtime failures.
• ⚡ Optimized for scale and efficiency: automatic batch size selection, distributed training (DDP, DeepSpeed), parameter efficient fine-tuning (PEFT), 4-bit quantization (QLoRA), paged and 8-bit optimizers, and larger-than-memory datasets.
• 📐 Expert level control: retain full control of your models down to the activation functions. Support for hyperparameter optimization, explainability, and rich metric visualizations.
• 🧱 Modular and extensible: experiment with different model architectures, tasks, features, and modalities with just a few parameter changes in the config. Think building blocks for deep learning.
• 🚢 Engineered for production: prebuilt Docker containers, native support for running with Ray on Kubernetes, export models to Torchscript and Triton, upload to HuggingFace with one command.

Ludwig is hosted by the Linux Foundation AI & Data.

Tech stack: Python 3.12 | PyTorch 2.6 | Pydantic 2 | Transformers 5 | Ray 2.54

💾 Installation

Install from PyPI. Be aware that Ludwig requires Python 3.12+.

pip install ludwig

Or install with all optional dependencies:

pip install ludwig[full]

Please see contributing for more detailed installation instructions.

🚂 Getting Started

Want to take a quick peek at some of Ludwig's features? Check out this Colab Notebook 🚀

Looking to fine-tune LLMs? Check out these notebooks:

1. Fine-Tune Llama-2-7b:
2. Fine-Tune Llama-2-13b:
3. Fine-Tune Mistral-7b:

For a full tutorial, check out the official getting started guide, or take a look at end-to-end Examples.

Large Language Model Fine-Tuning

Let's fine-tune a pretrained LLM to follow instructions like a chatbot ("instruction tuning").

Prerequisites

• HuggingFace API Token
• Access approval to your chosen base model (e.g., Llama-3.1-8B)
• GPU with at least 12 GiB of VRAM (in our tests, we used an Nvidia T4)

Running

We'll use the Stanford Alpaca dataset, which will be formatted as a table-like file that looks like this:

| instruction | input | output | | :-----------------------------------------------: | :--------------: | :-----------------------------------------------: | | Give three tips for staying healthy. | | 1.Eat a balanced diet and make sure to include... | | Arrange the items given below in the order to ... | cake, me, eating | I eating cake. | | Write an introductory paragraph about a famous... | Michelle Obama | Michelle Obama is an inspirational woman who r... | | ... | ... | ... |

Create a YAML config file named model.yaml with the following:

model_type: llm
base_model: meta-llama/Llama-3.1-8B

quantization:
  bits: 4

adapter:
  type: lora

prompt:
  template: |
    Below is an instruction that describes a task, paired with an input that may provide further context.
    Write a response that appropriately completes the request.

    ### Instruction:
    {instruction}

    ### Input:
    {input}

    ### Response:

input_features:
  - name: prompt
    type: text

output_features:
  - name: output
    type: text

trainer:
  type: finetune
  learning_rate: 0.0001
  batch_size: 1
  gradient_accumulation_steps: 16
  epochs: 3
  learning_rate_scheduler:
    decay: cosine
    warmup_fraction: 0.01

preprocessing:
  sample_ratio: 0.1

backend:
  type: local

And now let's train the model:

export HUGGING_FACE_HUB_TOKEN = "<api_token>"

ludwig train --config model.yaml --dataset "ludwig://alpaca"

Supervised ML

Let's build a neural network that predicts whether a given movie critic's review on Rotten Tomatoes was positive or negative.

Our dataset will be a CSV file that looks like this:

| movie_title | content_rating | genres | runtime | top_critic | review_content | recommended | | :------------------: | :------------: | :------------------------------: | :-----: | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | | Deliver Us from Evil | R | Action & Adventure, Horror | 117.0 | TRUE | Director Scott Derrickson and his co-writer, Paul Harris Boardman, deliver a routine procedural with unremarkable frights. | 0 | | Barbara | PG-13 | Art House & International, Drama | 105.0 | FALSE | Somehow, in this stirring narrative, Barbara manages to keep hold of her principles, and her humanity and courage, and battles to save a dissident teenage girl whose life the Communists are trying to destroy. | 1 | | Horrible Bosses | R | Comedy | 98.0 | FALSE | These bosses cannot justify either murder or lasting comic memories, fatally compromising a farce that could have been great but ends up merely mediocre. | 0 | | ... | ... | ... | ... | ... | ... | ... |

Download a sample of the dataset from here.

wget https://ludwig.ai/latest/data/rotten_tomatoes.csv

Next create a YAML config file named model.yaml with the following:

input_features:
  - name: genres
    type: set
    preprocessing:
      tokenizer: comma
  - name: content_rating
    type: category
  - name: top_critic
    type: binary
  - name: runtime
    type: number
  - name: review_content
    type: text
    encoder:
      type: embed
output_features:
  - name: recommended
    type: binary

That's it! Now let's train the model:

ludwig train --config model.yaml --dataset rotten_tomatoes.csv

Happy modeling

Try applying Ludwig to your data. Reach out on Discord if you have any questions.

❓ Why you should use Ludwig

• Minimal machine learning boilerplate

  Ludwig takes care of the engineering complexity of machine learning out of the box, enabling research scientists to focus on building models at