Alpha-gfn: Mining formulaic alpha factors with generative flow networks

---

Please check out our new paper that adopted the idea of this repo:

• :star2::star2::star2: Chen, B., Ding, H., Shen, N., Huang, J., Guo, T., Liu, L., & Zhang, M. (2025). AlphaSAGE: Structure-Aware Alpha Mining via GFlowNets for Robust Exploration.

Please cite this repository if you utilize any code provided herein.:

• APA style: Shen, N. (2024). Alpha-gfn: Mining formulaic alpha factors with generative flow networks. https://github.com/nshen7/alpha-gfn
• BibTeX style:

  @software{shen2024,
            author = {Shen, Ning},
            title = {Alpha-gfn: Mining formulaic alpha factors with generative flow networks},
            year = {2024},
            url = {https://github.com/nshen7/alpha-gfn}
        }
  

---

In this repo, we build an application that leverages a deep reinforcement learning framework to mine formulaic alpha factors using generative flow network models (i.e., GFlowNet). Due to industrial NDA, this repo only serves for demonstration purposes; hence, it only includes a simple example using a small amount of training data.

We give a brief introduction on the fundamental components of the project by answering the following questions.

• What are formulaic alpha factors and why search for them?
• Basic components of reinforcement learning？
• What are GFlowNet models?
• Why GFlowNet?

What are formulaic alpha factors and why search for them?

In quantitative investment, formulaic alpha factors are mathematical expressions or formulas used to identify and potentially exploit patterns or signals in financial data, for example, US stocks in this demo. These factors are typically derived from historical market data and are used to generate investment signals for trading strategies. Here are some common characteristics of formulaic alpha factors:

1. Quantitative Formulation: Alpha factors are expressed mathematically as formulas or algorithms that calculate a numerical value based on input feature data, such as stock prices, trading volumes, financial ratios, or other market indicators.
2. Signal Generation: Alpha factors are designed to capture signals or patterns in market data that are believed to be predictive of future price movements or other market dynamics. These signals may indicate opportunities for buying, selling, or holding securities.
3. Backtesting and Validation: Formulaic alpha factors are typically tested and validated using historical data to assess their effectiveness in generating positive returns or outperforming the market.
4. Integration into Trading Strategies: Alpha factors are often integrated into quantitative trading strategies, where they serve as inputs for decision-making processes, such as portfolio construction, position sizing, risk management, and trade execution.

In this project, we hope to develop an algorithm that generates assorted alpha factors that performs well in predicting the stock trend. In the most ideal case, they shall provide inspirations to alpha factor researchers or even be part of the trading strategies directly.

We define an alpha factor $f$ as a function mapping feature vectors of all stocks on a trading day $X_t$ into alpha values $z_t = f(X_t)$. Examples of formulaic alpha factors include moving averages, momentum indicators, relative strength indexes, price-to-earnings ratios, and various technical and fundamental indicators, etc. However, in this demo, <ins>we only consider technical indicators and our search space consists of daily frequency market data only</ins>, such as open and close price (see details in Methodology).

A common measure of effectiveness of an alpha factor is the (absolute) <ins>information correlation (IC)</ins> between the stock trend it aims to predict $y_t$ and the factor values $f(X_t)$, which is usually defined as the Spearman correlation of $y_t$ and $f(X_t)$.

Basic components of reinforcement learning？

Following is a non-exaustive list of important concepts in reinforcement learning, which are used in the project and will be mentioned in the rest of this introduction.

1. Agent: The learner or decision-maker that interacts with the environment. It takes actions and receives rewards based on its actions. In our case, it is the 'alpha generator' modeled by GFlowNet.
2. Environment: The external system with which the agent interacts. It receives actions from the agent and returns observations and rewards. In our case, it is the stock market.
3. State: A snapshot of the environment at a particular time. It contains all the relevant information necessary for decision-making. The state at time $t$ is usually denoted as $s_t$. In our case, it is the sequence of action tokens that forms the the mathematical expression of an alpha factor (see State space for details).
4. Action: A decision made by the agent that affects the state of the environment. The set of all possible actions is called the action space. The action at time $t$ is usually denoted as $a_t$. In our case, the action space consists of mathematical operators and market data features (see Action space for details).
5. Reward: A scalar feedback signal received by the agent from the environment. It indicates how good or bad the action taken by the agent was. The reward of state $s_t$ is usually dnoted as $R(s_t)$. In our case, it is defined as the square of information correlation (IC) penalized by the missing data proportion in the generated alpha factor (see Reward for details).
6. Policy: The strategy or rule that the agent uses to select actions based on states. It defines the mapping from states to actions. The probability function that describes the policy is usually denoted as $\pi(a_t|s_t)$. (See Policy for details.)

What are GFlowNet models?

A GFlowNet is a trained stochastic policy or generative model, trained such that it samples objects $x$ through a sequence of constructive steps (i.e., actions), with probability proportional to a reward function $R(x)$, where $R$ is a non-negative integrable function. After proper training sessions, a GFlowNet is expected to be able to sample a diversity of solutions $x$ that have a high value of $R(x)$. [1]

The word ‘flow’ in GFlowNet actually refers to unnormalized probabilities of an action given a state. The proposed approach views the probability assigned to an action given a state as the flow associated with a network whose nodes are states, and outgoing edges from that node are deterministic transitions driven by an action. The total flow into the network is the sum of the rewards in the terminal states (i.e., a partition function) and can be shown to be the flow at the root node (or start state). The proposed algorithm is inspired by Bellman updates and converges when the incoming and outgoing flow into and out of each state match. [4]

<div align="center"> <img src="misc/figures/gflownet_anim.gif" alt="Example Image" width=500> <figcaption>Source: [1] </figcaption> </div>

A neural net can be used to sample each of these forward-going constructive actions, one at a time. An object $x$ is done being constructed when a special "exit" action or a deterministic criterion of the state (e.g., $x$ has exactly $n$ elements) has been triggered, when we have reached a terminal state $x$, and after which we can get a reward $R(x)$.

PS: Reward can only be gained after reaching terminal state of a trajectory is the so-called episodic setting of RL.

<div align="center"> <img src="misc/figures/gfn-action.png" alt="Example Image"> <figcaption>Source: [1] </figcaption> </div>

Why GFlowNet?

GFlowNet is motivated by cases where diversity of the generated candidates is particularly important when the oracle is itself uncertain. An "oracle" typically refers to an idealized agent or system that has perfect knowledge of the environment and can provide optimal actions or solutions. Traditional reinforcement learning algorithms tries to generate the single highest-reward sequence of actions, while GFlowNet samples a distribution of trajectories whose probability is proportional to a given positive return or reward function. [4,5]

In the task of searching for formulaic alpha factors, not only the predictive performance of alpha factors are important, we also value the ability of exploration of the search algoithm due to the massive amount of stochasticity in stock markets. We may use IC as a metric of predictive performance but blindedly pursuing high absolue IC might cause overfitting and spurious correlation between the factors. Hence, GFlowNet is advantageous in our research problem where we want the sampled alpha factors are diversified.

Dataset

To alleviate computational burden, we use historical market data of stocks in the S&P 500 index from Year 2018 to Year 2019 for training purpose in this demo. Features include:

• Open - opening price
• High - maximum price during the day
• Low - minimum price during the day
• Close - close price adjusted for splits
• Volume - the number of shares that changed hands during a given day

Data sources:

• The set of stock ticks in S&P500 is extracted from https://github.com/datasets/s-and-p-500-companies/blob/main/data/constituents.csv.
• The US stock market data is downloaded from https://www.kaggle.com/datasets/jacksoncrow/stock-market-dataset.

You may find the pre-processing steps in notebooks/preprocess.ipynb.

Methodology <a name="methodology"></a>

The methodology is inspired by the framework proposed by [6]. Implementation of the methodology can be found in src/ folder. An example training session and some simple analyses can be found in notebooks/train.ipynb.

Action space <a name="action"></a>