HappyTorch

A PyTorch coding practice platform — covering LLM, Diffusion, PEFT, RLHF, and more

Like LeetCode, but for tensors. Self-hosted. Supports both Jupyter and Web interfaces. Instant auto-grading feedback. No GPU required.

中文版 README

News

• 2026-03-16: Thanks to damaoooo for reporting the notebook matching bug (attention vs multihead_attention). Replaced fragile suffix-based matching with an explicit name mapping for long-term reliability.
• 2026-03-13: Thanks to wavetao2010 for adding Docker image support with dual-mode (Web/Jupyter) and pre-built images.
• 2026-03-12: Web UI now groups problems by category (Fundamentals, Attention, RLHF, etc.) with collapsible sections in the sidebar for easier topic-based practice.
• 2026-03-10: Thanks to SongHuang1 for contributing the MLP XOR training problem (pure NumPy, manual forward + backward). Fixed Web UI issues: class-based tasks (LoRA, SwiGLU, etc.) now work correctly, added nn/F/numpy/math to execution namespace, fixed OpenMP crash on Windows, added MHA solution lookup, added 60s request timeout.
• 2026-03-09: Thanks to chaoyitud for adding ML and RLHF practice problems. Thanks to fiberproduct for fixing torch_judge/tasks/rope.py. Welcome everybody to contribute more problems!
• 2026-03-06: The plugin happytorch-plugin has been released.

---

Why HappyTorch?

If you're learning deep learning or preparing for ML interviews, you might have encountered these challenges:

• You've read many papers, but don't know where to start when it comes to implementing things from scratch
• You're asked to implement softmax or MultiHeadAttention in an interview, and your mind goes blank
• You want to deeply understand Transformer, LoRA, Diffusion, RLHF, but lack systematic practice

HappyTorch provides a friendly hands-on practice environment with 36 curated problems, from basic activation functions to complete Transformer components and RLHF algorithms.

| Feature | Description | |---------|-------------| | 36 curated problems | From basics to advanced, covering mainstream deep learning topics | | Auto-grading | Instant feedback showing what you got right and where to improve | | Two interfaces | LeetCode-like Web UI (Monaco Editor) or Jupyter notebooks | | Helpful hints | Get nudges when stuck, not full spoilers | | Reference solutions | Compare and learn after your own attempt | | Progress tracking | Record your learning journey |

---

Quick Start

# 1. Create and activate environment
conda create -n torchcode python=3.11 -y
conda activate torchcode

# 2. Install dependencies
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install jupyterlab numpy
pip install -e .

# 3. Prepare notebooks
python prepare_notebooks.py

# 4a. Launch Web Mode (recommended)
pip install fastapi uvicorn python-multipart
python start_web.py
# Open http://localhost:8000

# 4b. Or launch Jupyter Mode
python start_jupyter.py
# Open http://localhost:8888

Docker

# Web Mode (default, recommended)
make run                # build & start → http://localhost:8000
make stop               # stop container

# Jupyter Mode
make jupyter            # build & start → http://localhost:8888

# Or use the pre-built image directly (no build needed)
docker compose up -d                        # Web UI → http://localhost:8000
MODE=jupyter docker compose up -d           # Jupyter → http://localhost:8888

Progress data (data/progress.json) is persisted via Docker volume.

---

Web Mode

A LeetCode-like practice interface with:

• Monaco Editor — VS Code's editor with Python syntax highlighting
• Random / Sequential Mode — Get random unsolved problems or work through them in order
• Instant Testing — Run tests with one click (Ctrl+Enter)
• Solution Tab — View reference solutions with markdown explanation and copyable code
• Progress Dashboard — Track solved / attempted / todo status
• Dark Theme — Modern, eye-friendly interface

pip install fastapi uvicorn python-multipart
python start_web.py
# Open http://localhost:8000

---

Problem Set (36 Problems)

Fundamentals

| # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 1 | ReLU | relu(x) | | Activation functions, element-wise ops | | 2 | Softmax | my_softmax(x, dim) | | Numerical stability, exp/log tricks | | 3 | Linear Layer | SimpleLinear | | y = xW^T + b, Kaiming init, nn.Parameter | | 4 | LayerNorm | my_layer_norm(x, g, b) | | Normalization, affine transform | | 7 | BatchNorm | my_batch_norm(x, g, b) | | Batch vs layer statistics, train/eval | | 8 | RMSNorm | rms_norm(x, weight) | | LLaMA-style norm, simpler than LayerNorm |

Attention Mechanisms

| # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 5 | Scaled Dot-Product Attention | scaled_dot_product_attention(Q, K, V) | | softmax(QK^T/sqrt(d_k))V | | 6 | Multi-Head Attention | MultiHeadAttention | | Parallel heads, split/concat, projections | | 9 | Causal Self-Attention | causal_attention(Q, K, V) | | Autoregressive masking, GPT-style | | 10 | Grouped Query Attention | GroupQueryAttention | | GQA (LLaMA 2), KV sharing | | 11 | Sliding Window Attention | sliding_window_attention(Q, K, V, w) | | Mistral-style local attention | | 12 | Linear Attention | linear_attention(Q, K, V) | | Kernel trick, O(n*d^2) |

Full Architecture

| # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 13 | GPT-2 Block | GPT2Block | | Pre-norm, causal MHA + MLP, residual |

Modern Activation Functions (V2)

| # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 14 | GELU | gelu(x) | | Gaussian CDF, erf, BERT/GPT/DiT | | 15 | SiLU (Swish) | silu(x) | | x * sigmoid(x), LLaMA component | | 16 | SwiGLU | SwiGLU | | Gated activation, LLaMA MLP |

Parameter-Efficient Fine-Tuning (V2)

| # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 17 | LoRA | LoRALinear | | Low-rank BA, zero-init B, alpha/r scaling | | 18 | DoRA | DoRALinear | | Weight decomposition, magnitude + direction |

Conditional Modulation — Diffusion (V2)

| # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 19 | AdaLN | AdaLN | | Adaptive LayerNorm, DiT-style | | 20 | AdaLN-Zero | AdaLNZero | | Zero-init gate, stable training | | 21 | FiLM | FiLM | | Feature-wise modulation |

LLM Inference (V2)

| # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 22 | RoPE | apply_rotary_pos_emb(x, pos) | | Rotary embedding, 2D rotation | | 23 | KV Cache | KVCache | | Incremental caching for generation |

Diffusion Training (V2)

| # | Problem | Function / Class | Difficulty | Key Concepts | |:-:|---------|-----------------|:----------:|--------------| | 24 | Sigmoid Schedule | sigmoid_schedule(t, ...) | | S-curve noise schedule |

ML Fundamentals & Decoding (V3 — Community)

| # | Problem | Function /