Forest Through The Trees
Data Science Project: Replication of "Forest Through the Trees: Building Cross-Sections of Stock Returns" - creation of assets to test validity of factor models with Python
Install / Use
npx skills add Fernando-Urbano/forest-through-the-treesInstalls into whichever agent you are using.
README
Forest Through the Trees: Building Cross-Sections of Stock Returns Replication
Code Material
Through this code, I object to implement a replication of the paper "Forest Through the Trees: Building Cross-Sections of Stock Returns". The code is divided by its goals:
- portfolio constructions: has the functions to calculate RIDGE optimization, sorting, AP-Trees, etc.
- data: has the functions and databases to bring and organize characteristics, returns, and factors.
- results: has the results generated by the calculation.
The whole code can be reproduce in four simple steps:
- Assure the existence of the csv file "stock_characteristics_and_returns.csv" inside the "data" directory.
- Run "main.py" file to get the results for multiple time-frames, factor models, characteristics, amount of sorting quantiles, and depth of the AP-Tree.
- Run "results/results_graphics.py" to get the graphics of each experiment about (i) alpha t-value, (ii) Sharpe out-of-sample, (iii) R-squared.
- Run "results/join_graphics.py" to join the graphic results.
Abstract
This is replication is made to reinforce my interest in factor investing. The replication aims to create test assets using AP-Tree and compare the results of the test assets generated with the unconditional sorting. To compare the results, the test assets for both methods are used in a cross-section of returns to verify a capacity of a factor model to capture systematic risk and used to produce an out-of-sample Sharpe ratio, in order to attest the capacity to span the SDF. The test assets used are a crucial aspect when evaluating if a factor model captures all systematic risk and should be able to correctly span the SDF by representing all economic risk, and, therefore, have the highest out-of-sample Sharpe when combined efficiently. The results found in the original paper show that AP-Tree test assets considerably outperform unconditional sorting test assets in its capacity to span the SDF, due to its higher Sharpe ratio, and often lead to bigger and more significantly different from zero alpha in the cross-sections of returns, disproving the capacity of some factor models to price all systematic risk. The results found in this replication agree with results found on the original paper, but do not lead to an outperformance of the AP-Tree when compared to unconditional sorting test assets as big as the one found in the original work.
Disclaimers
Two major simplifications were made: (i) use of RIDGE steady of elastic net and mean shrinkage to produce the optimal mean-variance allocation, which permits the utilization of a close-end formula, (ii) use only of final nodes in the AP-Tree as test assets. Simplification number (ii) probably can be regarded as the major reason for difference in results: the new algorithm captures well the complex joint distribution of characteristics and avoids the curse of dimensionality by only dividing stocks into different groups when it helps to explain the excess of returns, nevertheless, only using final nodes leads to investment strategies with higher variance which do not necessarily reflect every time economic fundamentals. Simplification number (i) probably also declined the performance of the AP-Tree test assets, since the paper specifies that mean shrinkage often had a significant effect on the chosen portfolios spanning the SDF.
1. INTRODUCTION
1.1. THE QUESTION TO ANSWER: WHAT TEST ASSETS SHOULD BE USED TO EVALUATE A FACTOR MODEL?
The idea of the SDF is the basis of the whole factor investing industry, but it is not in the mind of every factor investor…
Nowadays, a zoo factor is present, and investors constantly find a risk-premium for what they happen to consider systematic risk. Nevertheless, finding a “risk-premium” or testing a new or old factor model is part of a scientific method, which requires finding a set of assets that can span the SDF, conditional on a given information set.
According to the idea behind the SDF, the price at any point in time of any asset can be calculated by the future cash flows discounted by a stochastic factor which accounts for the value of money in time (risk-free rate) and the non-diversifiable risk (systematic risk). Therefore, the difference between prices in two periods of time, in other words, the expected return, is the SDF, and can only be explained by the systematic risk and the risk-free rate. Thus, the excess of return can only be explained by systematic risk, a.k.a. exposition to factors. According to this view, being exposed to the right factors (maybe even at the right time) is the crucial aspect, which makes factor investing a continuous search for the model that succeeds in pricing the test assets; in order words, explain all the excess of return by the exposure of systematic risks, meaning that in a cross-section of betas (the exposure of an asset to a particular factor) explaining the excess of returns of an asset, the intercept (named “alpha”) should be non-significantly different from zero. Finding the factor model that can price all systematic risk is an arduous task, nevertheless, it is not the only. Before testing factor models, investors must decide which test assets to use. Employing test assets non-representative of the overall economy leads to not being able to correctly span the SDF.
When the SDF is not correctly spanned, a well-fitting factor model does not represent the truth since it does not account for all economic risk. In other words, the decision behind which test assets to use to verify the efficacy of a factor model to absorb the systematic risk is as crucial as deciding what factors should be considered systematic risks. For instance, if the first part is done wrongly, one “incomplete” factor model might not generate alpha, meaning that you would be prone to think that this “incomplete” model is able to capture all systematic risk, while a good set of test assets would clearly disprove the affirmative.
The idea behind spanning the SDF gained importance with the creation of multi-factor models. For instance, to create the FF3 model, Eugene Fama and Kenneth French realized that the market factor did not capture all systematic risks, since listed companies with smaller market caps systematically outperformed listed companies with bigger ones, and listed companies with bigger book-to-market ratios systematically outperformed listed companies with smaller ones. Therefore, to scientifically evaluate the performance of the CAPM, they proposed the construction of long-short portfolios of stocks based on the characteristics previously described. They regressed the portfolios by the market index, and finally constructed a cross-section with the market betas of those portfolios, in which the market attempts to explain the excess of return. The result was clear: the expected returns of those portfolios could not be fully explained by the market on a considerable set of cross-sections. That is, portfolios that accounted for more economic risk were able to disprove the affirmative that the market contained all systematic risks. The result of Fama and French was largely due to the capacity to look at a different set of assets and generate better test assets than the ones considered before.
Nevertheless, their test assets were not perfect…
1.2. CONDITIONAL VS. UNCONDITIONAL SORTING
In this paper replication, we explain and reproduce conventional unconditional sorting, the same method used by Fama and French, and the new conditional sorting method: AP-Tree. During the explanation of the AP-Tree we point out the problems of the old method and address the simplifications done in comparison to the complete article.
1.2.1. TIME SERIES AND CROSS-SECTIONS: HOW TO TEST THE PRICING CAPACITY OF A FACTOR MODEL?
Before entering in conditional and unconditional sorting, we jump to the end of the experiment, which is the test of a factor model, so later we can focus only on the main team. The final part of the experiment for a given pool of test assets is composed of:
- i) Regress the excess return of each one of the test assets by the returns of the factors in the out-of-sample (time after the portfolio creation);
- ii) Capture the betas of the factors for each one of the test assets and use each of the vectors of factor betas as a feature, where the number of observations for each is the number of tests assets;
- iii) Use those betas as features in a cross-section regression explaining the accumulated excess of return of the test assets for the period;
- iv) Check the presence of an intercept, which is called alpha, and if it is significantly different from zero.
If the alpha of the cross-section regression is non-significantly different from zero, one of two options will be true: (i) the factor model captures all systematic risk for this period, or (ii) the test assets used did not span the SDF well and do not represent all economic non-diversifiable risk, creating misleading results.
After applying unconditional and conditional sorting to find the test assets, we do the four steps described above.
1.2.2. DESIRABLE CHARACTERISTICS OF PORTFOLIOS SPANNING THE SDF
As seen before, the non-optimal spanning of the SDF leads to unreliable results. That is why correctly spanning the SDF, according to the paper, should:
i) reflect the impact of multiple characteristics at the same time. ii) achieve the highest possible Sharpe Ratio out-of-sample when combined together. iii) allow for a flexible nonlinear dependence and interactions. iv) be a relatively small number of well-diversified managed portfolios feasible for investors. v) provide an interpretable link to fundamentals.
1.2.3. UNCONDITIONAL SORTING
The unconditional method, used by Fama and French, uses quantiles of characteristics to divide stocks into groups and later create portfolios. For instance, in “triple sort
Related Skills
node-connect
385.5kDiagnose OpenClaw Android, iOS, or macOS node pairing, QR/setup code, route, auth, and connection failures.
python-debugpy
385.5kDebug Python with pdb, breakpoint(), post-mortem inspection, and debugpy remote attach.
skill-creator
385.5kCreate, edit, audit, tidy, validate, or restructure AgentSkills and SKILL.md files.
claude-opus-4-5-migration
140.6kMigrate prompts and code from Claude Sonnet 4.0, Sonnet 4.5, or Opus 4.1 to Opus 4.5
