TensorTrade

Train RL agents to trade. Can they beat Buy-and-Hold?

TensorTrade is an open-source Python framework for building, training, and evaluating reinforcement learning agents for algorithmic trading. The framework provides composable components for environments, action schemes, reward functions, and data feeds that can be combined to create custom trading systems.

Quick Start

# Requires Python 3.12+
python3.12 -m venv tensortrade-env && source tensortrade-env/bin/activate
pip install -e .

# For training with Ray/RLlib (recommended)
pip install -r examples/requirements.txt

# Run training
python examples/training/train_simple.py

Documentation & Tutorials

📚 Tutorial Index — Start here for the complete learning curriculum.

Foundations

The Three Pillars — RL + Trading + Data concepts
Architecture — How components work together
Your First Run — Run and understand output

Domain Knowledge

Trading for RL Practitioners
RL for Traders
Common Failures — Critical pitfalls to avoid
Full Introduction — New to both domains

Core Components

Action Schemes — BSH and order execution
Reward Schemes — Why PBR works
Observers & Feeds — Feature engineering

Training

First Training — Train with Ray RLlib
Ray RLlib Deep Dive — Configuration options
Optuna Optimization — Hyperparameter tuning

Advanced Topics

Overfitting — Detection and prevention
Commission Analysis — Key research findings
Walk-Forward Validation — Proper evaluation

Additional Resources

Experiments Log — Full research documentation
Environment Setup — Detailed installation guide
API Reference

Research Findings

We conducted extensive experiments training PPO agents on BTC/USD. Key results:

| Configuration | Test P&L | vs Buy-and-Hold | |---------------|----------|-----------------| | Agent (0% commission) | +$239 | +$594 | | Agent (0.1% commission) | -$650 | -$295 | | Buy-and-Hold | -$355 | — |

The agent demonstrates directional prediction capability at zero commission. The primary challenge is trading frequency—commission costs currently exceed prediction profits. See EXPERIMENTS.md for methodology and detailed analysis.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        TradingEnv                               │
│                                                                 │
│   Observer ──────> Agent ──────> ActionScheme ──────> Portfolio │
│   (features)      (policy)      (BSH/Orders)        (wallets)  │
│       ^                                                  │      │
│       └──────────── RewardScheme <───────────────────────┘      │
│                        (PBR)                                    │
│                                                                 │
│   DataFeed ──────> Exchange ──────> Broker ──────> Trades       │
└─────────────────────────────────────────────────────────────────┘

| Component | Purpose | Default | |-----------|---------|---------| | ActionScheme | Converts agent output to orders | BSH (Buy/Sell/Hold) | | RewardScheme | Computes learning signal | PBR (Position-Based Returns) | | Observer | Generates observations | Windowed features | | Portfolio | Manages wallets and positions | USD + BTC | | Exchange | Simulates execution | Configurable commission |

Training Scripts

| Script | Description | |--------|-------------| | examples/training/train_simple.py | Basic demo with wallet tracking | | examples/training/train_ray_long.py | Distributed training with Ray RLlib | | examples/training/train_optuna.py | Hyperparameter optimization | | examples/training/train_best.py | Best configuration from experiments |

Installation

Requirements: Python 3.11 or 3.12

# Create environment
python3.12 -m venv tensortrade-env
source tensortrade-env/bin/activate  # Windows: tensortrade-env\Scripts\activate

# Install
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .

# Verify
pytest tests/tensortrade/unit -v

# Training dependencies (optional)
pip install -r examples/requirements.txt

See ENVIRONMENT_SETUP.md for platform-specific instructions and troubleshooting.

Docker

make run-notebook  # Jupyter
make run-docs      # Documentation
make run-tests     # Test suite

Project Structure

tensortrade/
├── tensortrade/           # Core library
│   ├── env/              # Trading environments
│   ├── feed/             # Data pipeline
│   ├── oms/              # Order management
│   └── data/             # Data fetching
├── examples/
│   ├── training/         # Training scripts
│   └── notebooks/        # Jupyter tutorials
├── docs/
│   ├── tutorials/        # Learning curriculum
│   └── EXPERIMENTS.md    # Research log
└── tests/

Troubleshooting

| Issue | Solution | |-------|----------| | "No stream satisfies selector" | Update to v1.0.4-dev1+ | | Ray installation fails | Run pip install --upgrade pip first | | NumPy version conflict | pip install "numpy>=1.26.4,<2.0" | | TensorFlow CUDA issues | pip install tensorflow[and-cuda]>=2.15.1 |

Contributing

See CONTRIBUTING.md for guidelines.

Priority areas:

Trading frequency reduction (position sizing, holding periods)
Commission-aware reward schemes
Alternative action spaces

Community

License

Apache 2.0

Tensortrade

Install / Use

README