TorchTrade

A machine learning framework for algorithmic trading built on TorchRL.

TorchTrade's goal is to provide accessible deployment of RL methods to trading. The framework supports various RL methodologies including online RL, offline RL, model-based RL, contrastive learning, and many more areas of reinforcement learning research. Beyond RL, TorchTrade integrates traditional trading methods such as rule-based strategies, as well as modern approaches including LLMs (both local models and frontier model integrations) as trading actors.

TorchTrade provides modular environments for both live trading with major exchanges and offline backtesting. The framework supports:

🎯 Multi-Timeframe Observations - Train on 1m, 5m, 15m, 1h bars simultaneously
🤖 Multiple RL Algorithms - PPO, DQN, IQL, GRPO, DSAC, CTRL implementations
📊 Feature Engineering - Add technical indicators and custom features
🔴 Live Trading - Direct Alpaca, Binance, Bitget, and Bybit API integration
🧠 LLM Integration - Use GPT-4o-mini or local LLMs as trading agents
📐 Rule-Based Actors - Hard-coded strategies for imitation learning and baselines
🔮 Pretrained Encoder Transforms - Foundation model embeddings for time series
📦 Ready-to-Use Datasets - Pre-processed OHLCV data at HuggingFace/Torch-Trade
📈 Research to Production - Same code for backtesting and live deployment
🛠️ Claude Code Agents - Pre-built AI coding agents for developing and extending TorchTrade (available on our website)
📝 Research Articles - In-depth articles on RL trading strategies and framework design (get here)

⚠️ Work in Progress: TorchTrade is under active development. We continuously add new features, improvements, and optimizations. Expect API changes, new environments, and enhanced functionality in future releases.

Current Scope: The framework currently focuses on single-asset trading environments (one symbol per environment). Multi-asset portfolio optimization and cross-asset trading environments are planned for future releases.

📚 Website & Documentation

🌐 TorchTrade Website — Landing page with overview, features, Claude Code agents, and research articles

📖 TorchTrade Documentation — Comprehensive guides, tutorials, and API reference

Getting Started - Installation and first environment
Environments - Offline and online trading environments
Examples - Training scripts for PPO, IQL, GRPO, and more
Components - Loss functions, transforms, and actors
Advanced Customization - Custom features, rewards, and environments

Quick Start

1. Installation

# Install UV (fast Python package installer)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Clone and install
git clone https://github.com/TorchTrade/torchtrade.git
cd torchtrade
uv sync
source .venv/bin/activate  # On Unix/macOS

# Optional: Install with extra features
uv sync --extra llm              # LLM actors (OpenAI API + local vLLM/transformers)
uv sync --extra chronos          # Chronos forecasting transforms
uv sync --all-extras             # Install all optional dependencies

2. Your First Environment

from torchtrade.envs.offline import SequentialTradingEnv, SequentialTradingEnvConfig
import pandas as pd

# Load OHLCV data
df = pd.read_csv("btcusdt_1m.csv")
df['timestamp'] = pd.to_datetime(df['timestamp'])

# Create environment (spot trading = long-only)
config = SequentialTradingEnvConfig(
    trading_mode="spot",  # or "futures" for leveraged trading
    time_frames=["1min", "5min", "15min"],
    window_sizes=[12, 8, 8],
    execute_on=(5, "Minute"),
    initial_cash=1000
)
env = SequentialTradingEnv(df, config)

# Run
tensordict = env.reset()
tensordict = env.step(tensordict)
print(f"Reward: {tensordict['reward'].item()}")

3. Train Your First Policy

# Train PPO with default settings
uv run python examples/online_rl/ppo/train.py

# Customize with Hydra overrides
uv run python examples/online_rl/ppo/train.py \
    env.symbol="BTC/USD" \
    optim.lr=1e-4

For detailed tutorials, see Getting Started Guide.

Live Environments

TorchTrade supports live trading with major exchanges:

| Environment | Exchange | Asset Type | Futures | Leverage | Bracket Orders | |-------------|----------|------------|---------|----------|----------------| | AlpacaTorchTradingEnv | Alpaca | Crypto/Stocks | ❌ | ❌ | ❌ | | AlpacaSLTPTorchTradingEnv | Alpaca | Crypto/Stocks | ❌ | ❌ | ✅ | | BinanceFuturesTorchTradingEnv | Binance | Crypto | ✅ | ✅ (1-125x) | ❌ | | BinanceFuturesSLTPTorchTradingEnv | Binance | Crypto | ✅ | ✅ (1-125x) | ✅ | | BitgetFuturesTorchTradingEnv | Bitget | Crypto | ✅ | ✅ (1-125x) | ❌ | | BitgetFuturesSLTPTorchTradingEnv | Bitget | Crypto | ✅ | ✅ (1-125x) | ✅ | | BybitFuturesTorchTradingEnv | Bybit | Crypto | ✅ | ✅ (1-100x) | ❌ | | BybitFuturesSLTPTorchTradingEnv | Bybit | Crypto | ✅ | ✅ (1-100x) | ✅ |

Need another broker? Request support for additional platforms (OKX, Interactive Brokers, etc.) by creating an issue or emailing torchtradecontact@gmail.com.

See Online Environments Documentation for setup guides and examples.

Trading Platforms

Start live trading with these supported platforms:

🪙 Cryptocurrency Trading

Binance - Leading cryptocurrency exchange

Supported by: BinanceFuturesTorchTradingEnv, BinanceFuturesSLTPTorchTradingEnv
Features: Spot & futures trading, up to 125x leverage, testnet available
Commission: Maker 0.02% / Taker 0.04% (with BNB discount)
Get Started: Sign up for Binance

Bitget - Fast-growing cryptocurrency exchange

Supported by: BitgetFuturesTorchTradingEnv, BitgetFuturesSLTPTorchTradingEnv
Features: Futures trading with up to 125x leverage, testnet for safe testing
Commission: Maker 0.02% / Taker 0.06%
Get Started: Sign up for Bitget

Bybit - Top cryptocurrency derivatives exchange

Supported by: BybitFuturesTorchTradingEnv, BybitFuturesSLTPTorchTradingEnv
Features: Futures trading with up to 100x leverage, native bracket orders (SL/TP), testnet for safe testing
Commission: Maker 0.02% / Taker 0.055%
Get Started: Sign up for Bybit

📈 Stock & Crypto API

Alpaca - Commission-free trading API

Supported by: AlpacaTorchTradingEnv, AlpacaSLTPTorchTradingEnv
Features: Commission-free stocks & crypto, paper trading, real-time data
Best for: US markets, algorithmic trading
Get Started: Sign up for Alpaca

Support TorchTrade Development

Buy Me a Coffee: buymeacoffee.com/torchtrade
⭐ Star the repo: Help others discover TorchTrade on GitHub

Your support helps maintain the project, add new features, and keep documentation up-to-date!

<details> <summary><h2>📦 Offline Environments</h2></summary>

All environments support both spot (leverage=1) and futures (leverage>1) trading via config.

| Environment | Bracket Orders | One-Step | Best For | |-------------|----------------|----------|----------| | SequentialTradingEnv | ❌ | ❌ | Standard sequential trading | | SequentialTradingEnvSLTP | ✅ | ❌ | Risk management with SL/TP | | OneStepTradingEnv | ✅ | ✅ | GRPO, contextual bandits |

See Offline Environments Documentation for detailed guides.

</details> <details> <summary><h2>🚀 Training Algorithms & Examples</h2></summary>

TorchTrade includes implementations of multiple RL algorithms, all usable across any environment via Hydra config switching:

PPO - examples/online_rl/ppo/
PPO + Chronos (time series embeddings) - examples/online_rl/ppo_chronos/
DQN - examples/online_rl/dqn/
IQL - examples/online_rl/iql/
DSAC - examples/online_rl/dsac/
GRPO - examples/online_rl/grpo/
CTRL - Research

Run Training Examples

# PPO with default environment (sequential SLTP)
uv run python examples/online_rl/ppo/train.py

# PPO with different environments (switch via command-line)
uv run python examples/online_rl/ppo/train.py env=sequenti

Torchtrade

Install / Use

README