<br> <br>

HFT Framework

This repository is home to a High-Frequency Trading (HFT) framework, developed using Java and Python, primarily for research applications. The framework is engineered to interface with live markets through the use of connectors, which can be integrated within the same process or remotely via the ZeroMQ networking library.

A significant feature of this framework is its ability to perform backtesting at the L2 tick data level, utilizing the same codebase as that used for live market interfacing. This capability allows for a detailed and granular analysis of trading strategies, providing valuable insights into their potential performance in live markets.

Feedback, suggestions, and modifications are welcomed and appreciated.<br> <br>

Please note: This framework has not been validated in a live trading environment. Proceed with caution and assume all associated risks. <br> <br>

<!-- TOC -->

• HFT Framework
  • How-to use
    • 1. Create algorithm and backtest
      • 1.1 Java Algorithms
      • 1.2 Pure Python Strategies (python_algo)
    • 2. Live trading
      • Monitoring
    • Web Monitoring UI
    • 3. Market Engine
      • XChangeEngine
      • MetatraderEngine
  • Arquitecture
    • Backtest
    • Live Paper/Staging trading
    • Live trading
  • Environment settings
    • Optional
  • I owe you one
  • TODO
    • Java documentation reference
    • Monitoring
    • Reference

<!-- TOC -->

How-to use

LLM Documentation

1. Create algorithm and backtest

There is a standalone project for creation a custom algorithm in this github repository

• ALGORITHM_DOCUMENTATION.md
• BACKTEST_DOCUMENTATION.md
• MARKET_MAKING_ALGORITHMS_DOCUMENTATION.md
• MONITORING_DOCUMENTATION.md

1.1 Java Algorithms

In this instance, we execute a backtest for the Java strategies ConstantSpread and LinearConstantSpread. These instructions pertain to the execution of pre-existing algorithms.

To develop a new algorithm, one must create a new class that extends from Algorithm.java and incorporate it into the trading algorithms provider method getAlgorithm in TradingAlgorithmsProvider.java

1. Execute the compilation and packaging process for the Backtest module, which will result in the generation of a JAR file. The target location for this file is java/executables/Backtest/target/Backtest.jar.If you want to include your own algorithms you can create your own private_trading_algorithms module
2. Establish a reference to the aforementioned path in the environment variable denoted as LAMBDA_JAR_PATH.
3. Ensure the data folder is prepared and contains the necessary Parquet files for the backtest. An example data set is provided for reference.
4. Establish a reference to the data path in the environment variable denoted as LAMBDA_DATA_PATH.
5. Initiate the backtest process. This can be achieved through one of the available options.
   • Java: configuring json ConstantSpread backtest java -jar Backtest.jar example_ConstantSpread.json
   • Python: like in the ConstantSpread example ConstantSpread

   constant_spread = ConstantSpread(algorithm_info='test_main')
   output_test = constant_spread.test(
           instrument_pk='btcusdt_kraken',
           start_date=datetime.datetime(year=2023, day=13, month=11, hour=9),
           end_date=datetime.datetime(year=2023, day=13, month=11, hour=15),
       )
   

1.2 Pure Python Strategies (python_algo)

The framework supports pure-Python trading strategies that communicate with the Java framework via ZeroMQ. This allows you to write strategies entirely in Python while leveraging the Java backtesting and live trading infrastructure.

Architecture:

• Java PUB → Python SUB: Market data events (depth, trade, execution reports, candles)
• Java PULL ← Python PUSH: Order/quote commands (asynchronous)
• Java REP ↔ Python REQ: Synchronous requests (portfolio snapshot, etc.)

Transport Options:

• TCP (default): Works across hosts, localhost:7700-7703
• IPC: Same-host only, lower latency via Unix domain sockets

Codec Options:

• JSON (default): Human-readable, always available
• MessagePack: ~3× faster parsing, smaller frames

Quick Start:

from python_algo import PythonStrategy, ZmqTransport, DepthMsg, TradeMsg, ExecutionReportMsg, CandleMsg, OrderRequestCmd

class MyStrategy(PythonStrategy):
    def on_depth(self, depth: DepthMsg) -> None:
        if depth.spread < 0.01:
            self.send_order(OrderRequestCmd(
                instrument=depth.instrument,
                verb="Buy",
                order_type="Limit",
                quantity=0.01,
                price=depth.best_bid
            ))
    
    def on_trade(self, trade: TradeMsg) -> None:
        pass
    
    def on_execution_report(self, er: ExecutionReportMsg) -> None:
        print(f"Order {er.status}: {er.verb} {er.quantity} @ {er.price}")
    
    def on_candle(self, candle: CandleMsg) -> None:
        pass

# TCP + JSON (default)
transport = ZmqTransport(md_sub_port=7700, cmd_push_port=7701, req_port=7703)
strategy = MyStrategy(transport, instruments=["btcusdt_binance"])
strategy.run()

Java Configuration (PythonAlgorithm):

To run a Python strategy, configure the Java side to use PythonAlgorithm:

{
  "algorithm": {
    "algorithmName": "PythonAlgorithm",
    "algorithmParameters": {
      "python_transport_type": "tcp",
      "python_md_pub_port": "7700",
      "python_cmd_pull_port": "7701",
      "python_rep_port": "7703",
      "python_codec": "json",
      "python_backtest_sync": "false"
    }
  },
  "instruments": ["btcusdt_binance"],
  "startDate": "2023-11-13 09:00:00",
  "endDate": "2023-11-13 15:00:00"
}

Parameters:

• python_transport_type: "tcp" (default) or "ipc"
• python_md_pub_port: Port for market data (default: 7700)
• python_cmd_pull_port: Port for commands (default: 7701)
• python_rep_port: Port for synchronous requests (default: 7703)
• python_codec: "json" (default) or "msgpack"
• python_backtest_sync: Enable ACK handshake for debugger-friendly backtesting (default: false)
• python_host: Bind address for TCP mode (default: "*")
• python_ipc_md_path: IPC socket path for market data (default: "/tmp/python_algo_md")
• python_ipc_cmd_path: IPC socket path for commands (default: "/tmp/python_algo_cmd")
• python_ipc_rep_path: IPC socket path for requests (default: "/tmp/python_algo_req")

Synchronous Portfolio Snapshot:

class MyStrategy(PythonStrategy):
    def on_depth(self, depth: DepthMsg) -> None:
        # Request current portfolio state
        snapshot = self.get_portfolio_snapshot(timeout_ms=5000)
        
        if snapshot:
            print(f"Total P&L: {snapshot.total_pnl:.2f}")
            print(f"Net Position: {snapshot.net_position:.4f}")
            
            # Per-instrument breakdown
            for instrument, pnl in snapshot.instrument_pnl_snapshots.items():
                print(f"{instrument}: {pnl}")

Examples:

The python/python_algo/examples directory contains complete working examples:

1. avellaneda_stoikov_strategy.py - Market making with dynamic spreads

   # Run with backtest:
   python python/python_algo/examples/run_alpha_as_backtest.py
   
   # Run with live ZeroMQ:
   python python/python_algo/examples/run_alpha_as_zeromq.py
   

2. sma_candle_strategy.py - Simple Moving Average crossover on candles

   # Run with backtest:
   python python/python_algo/examples/run_sma_backtest.py
   
   # Run with live ZeroMQ:
   python python/python_algo/examples/run_sma_candle_zeromq.py
   

3. **[test_portfolio_snapshot.py](python/python_algo/