Candle Breaks Project

Come hang out in the Fib Lab Discord — questions, ideas, or feedback welcome: https://discord.gg/uY9vkJpZxb

This project is, at least initially, a Backtesting Starter Kit for the Candle Breaks method.

What's New

• Downloader script optimized; can now update historical data to current in ~15 seconds if run frequently.
• New script historical_instances_finder_updater.py replaces the previous finder scripts; built with the same incremental approach as the downloader. It checks what instances already exist and scans candles from the last instance forward, adding only what’s new. It detects 1v1s and 1v1+1s, and will upgrade the last instance from the previous run to 1v1+1 if appropriate. It can also run in ~15 seconds if run frequently.
• Minor tweaks to the simulator code, mostly fixing some bugs in dates shown in reports.
• New Pine script Pine Scripts/Candle Breaks 1v1 + 2v2 by oneofus2233.txt handles both 1v1s and 2v2s. Thank you to @oneofus2233 for submitting! Note: alert functions were not tested; display works perfectly here. Based on the 1.2 version of the original script.

Project Structure

The Python folder is organized into:

• Processing: Contains scripts for fetching historical candle data, converting timeframes, finding instances, and processing instance data.
• Analysis: A place to contain scripts for analyzing instances, with a sample script.
• BotSim1.0: Contains a program for simulating trading strategies based on processed instances.
• Airtable: Contains the Airtable Updater script.
• Plotter tool is not ready for this release and will be coming soon.

Required Python Libraries

• ccxt
• pandas
• tqdm

You can install all the required libraries using the following pip command:

pip install ccxt pandas tqdm

A Quick Note on Paths

The Github comes with an empty set of folders in the Data folder as a suggested structure to hold historical data and all of the output of the various scripts. Nearly all of these scripts have one or more input and output folder paths that by default make use of the suggested structure. You can specify paths by changing the default listed near the top of the script (and just press enter when prompted), or you can copy/paste the path into the program when it prompts you to do so. This is what one of the default path variables look like:

default_folder_path = os.path.join("..", "..", "Data")

Using os.path.join allows us to create a path that works on both Windows (where \ is used) or Linux/Mac (where / is used). You can string together as many folders as you want by adding more commas and folder names in quotation marks inside the brackets.

We're also using relative paths here, where ".." means up one level from where the script is run. In most cases we go up two levels to find the Data folder.

Processing Folder

The scripts in the Processing folder are used for fetching historical candle data, converting timeframes, finding instances, and processing instance data.

Step 1: Download historical data. Run download_binance_historical_data.py

This script downloads historical candle data for any trading pair (default: SOLUSDT) from Binance and saves it in CSV format.

Command Line Arguments

The script now accepts command line arguments, and has three operation modes:

1. --some Mode:

   • Downloads all standard timeframes directly from Binance (Standard timeframes are: 1m, 5m, 15m, 30m, 1h, 2h, 4h, 6h, 8h, 12h, 1d, 3d, 1w, 1mo)
   • Overwrites any existing candle data files
   • Useful if you only want to do backtesting on a small set of timeframes.
   • Example: python download_binance_historical_data.py --some

2. --all Mode:

   • Downloads only the 1m timeframe data from Binance
   • Then automatically launches the timeframe converter script to generate all other timeframes. You can customize the list of timeframes by editing the converter script.
   • Useful for creating a complete set of custom timeframes efficiently
   • Example: python download_binance_historical_data.py --all

3. Default Mode (neither --all nor --some):

   • First processes the 1m timeframe. If it does not exist, it will download it in full from Binance. If it does exist, it will just download enough data to bring it up to current.
   • Then it updates all other timeframe files found in the same folder using data from the 1m timeframe
   • Efficient approach that only downloads necessary data to update files
   • Example: python download_binance_historical_data.py

Additional Command Line Options:

• -s, --symbol SYMBOL: Specify trading pair (default: SOLUSDT)

  • Example: python download_binance_historical_data.py -s BTCUSDT

• -k, --keep-incomplete: Keep incomplete candles

  • By default, the script will delete any candles that represent an incomplete period of time. If you don't want the script to do that, you can specify the -k argument to keep these incomplete candles.
  • Example: python download_binance_historical_data.py -k

• -d, --directory PATH: Set custom directory for candle data

  • Not recommended; only use this if you don't want to use the suggested folder structure.
  • Example: python download_binance_historical_data.py -d "C:\MyCandleData"

• --start-date DATE: Set start date for historical data (YYYY-MM-DD or YYYYMMDD)

  • Example: python download_binance_historical_data.py --start-date 2022-01-01

• --end-date DATE: Set end date for historical data (YYYY-MM-DD or YYYYMMDD, default: current date)

  • Example: python download_binance_historical_data.py --end-date 2023-12-31

• -v, --verbose: Enable detailed output during processing

  • Example: python download_binance_historical_data.py -v

(Optional) Step 2: Create custom timeframes. Run historical_data_TF_converter.py

• This script resamples the 1m candle data into specified custom timeframes and saves the resulting data as new CSV files.
• The default list of custom TFs was provided by syndotc. The intra-day TFs are all factors of 1440, or in other words, divide evenly into a day. The multi-day TFs run up to 36D. Also included are the 1W up to 6W, and the 1mo and 2mo.
• This script will now optionally accept -p or --path as a command line argument, followed by a path to your folder where the 1m candle data is saved.

Step 3: Find 1v1 and 1v1+1 instances. Run historical_instances_finder_updater.py

• Scan historical candle data to identify 1v1 and 1v1+1 candle breaks and write them to instance CSVs.
• Computes Fibonacci retrace levels for further analysis and saves them with the instances.
• Reads what’s already in your files and adds new instances going forward; if the latest prior instance qualifies as 1v1+1, it will upgrade that entry.
• Recommended as the standard finder (supersedes the older 1v1 finder scripts).

(Optional) Step 4: Find XvY instances. Run historical_opps_finder_XvY.py

• Process historical candle data to find XvY candle breaks.
• Computes Fibonacci retrace levels for further analysis and saves the results to CSV files.

Step 5: Process instances.

Run historical_process_status_of_instances.py

• This script processes instances using the 1m candle data as a price reference.
• It calculates whether specific price targets or Fibonacci levels are reached and updates the instance's status accordingly. It also records timestamps of the various fib levels being reached.
• The processed files are saved to an output folder, and the original files can be optionally deleted.
• This "single core" version of the file will do all processing in one thread, which may take a long time.

Analysis Folder

The Analysis folder contains scripts for analyzing processed instances.

TF_Instances_Summarizer.py

• This script produces a summary of instances by timeframe.
• It calculates various metrics such as counts of longs/shorts, win rates, and average times for different stages of fib levels.

BotSim1.0

The BotSim1.0 folder contains a program for simulating trading strategies based on processed instances. The simulator offers numerous configuration options (available in config.py) to fine-tune your trading strategy. It produces the following output:

• A minute-by-minute tracking of total bankroll, long / short position size, unrealized PnL, and cost basis. This is split into monthly files named analysis_yyyymm.csv
• A list of currently open positions in open_positions.csv. As trades are closed, they are removed from this file and moved to...
• A list of closed positions in closed_positions.csv. This file contains instance performance, because each row represents an open trade and a close trade.
• A list of all trades made in four types: open long, close long, open short, close short. This file is called trades_all.csv.
• A performance summary report, named (folder name)_(start_date)_(end_date).csv. It produces a monthly performance summary, and a report on results by timeframe.

Brief description of the files:

• config.py - Contains the configurable settings for a simulator run.
• main.py - Launch this to start the program. Contains the main menu.
• initialization.py - Contains code to load state data and resume a previously started run.
• log_utils.py / reporting.py - Contains logging utilities and generates the summary data file.
• sim_entries.py - Contains logic for executing entries during the sim.
• sim_exits.py - Contains logic for executing exits during the sim.
• position_size.py - Contains logic for determining the position size of an entry.
• simulation.py - Contains the main structure of the simulation processing loop.

Simulator Features

The simulator offers a wide range of configurable