Orchestra-Research

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques

Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss

Expert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support

Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss

Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction

GGUF format and llama.cpp quantization for efficient CPU/GPU inference

Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.

Provides guidance for enterprise-grade RL training using miles, a production-ready fork of slime

Open-source AI observability platform for LLM tracing, evaluation, and monitoring

Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO.

Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.

Provides guidance for PyTorch-native agentic RL using torchforge, Meta's library separating infra from algorithms

Provides guidance for training LLMs with reinforcement learning using verl (Volcano Engine RL)

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching

Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent with full horsepower. Maintained by Orchestra Research.

Expert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support