vecstack

Python package for stacking (stacked generalization) featuring lightweight functional API and fully compatible scikit-learn API
Convenient way to automate OOF computation, prediction and bagging using any number of models

• Functional API:
  • Minimalistic. Get your stacked features in a single line
  • RAM-friendly. The lowest possible memory consumption
  • Kaggle-ready. Stacked features and hyperparameters from each run can be automatically saved in files. No more mess at the end of the competition. Log example
• Scikit-learn API:
  • Standardized. Fully scikit-learn compatible transformer class exposing fit and transform methods
  • Pipeline-certified. Implement and deploy multilevel stacking like it's no big deal using sklearn.pipeline.Pipeline
  • And of course FeatureUnion is also invited to the party
• Overall specs:
  • Use any sklearn-like estimators
  • Perform classification and regression tasks
  • Predict class labels or probabilities in classification task
  • Apply any user-defined metric
  • Apply any user-defined transformations for target and prediction
  • Python 3.9+, unofficial support for Python 2.7 and 3.4
  • Win, Linux, Mac
  • MIT license
  • Depends on numpy, scipy, scikit-learn>=1.6.0

Get started

• FAQ
• Installation guide
• Usage:
  • Functional API
  • Scikit-learn API
• Tutorials:
  • Stacking concept + Pictures + Stacking implementation from scratch
• Examples (all examples are valid for both API with little difference in parameters):
  • Functional API:
    • Regression
    • Classification with class labels
    • Classification with probabilities + Detailed workflow
  • Scikit-learn API:
    • Regression + Multilevel stacking using Pipeline
• Documentation:
  • Functional API or type >>> help(stacking)
  • Scikit-learn API or type >>> help(StackingTransformer)

Installation

Note: Python 3.9+ is officially supported and tested. If you’re still using Python 2.7 or 3.4 see installation details here

• Classic 1st time installation (recommended):
  • pip install vecstack
• Install for current user only (if you have some troubles with write permission):
  • pip install --user vecstack
• If your PATH doesn't work:
  • /usr/bin/python -m pip install vecstack
  • C:/Python3/python -m pip install vecstack
• Upgrade vecstack and all dependencies:
  • pip install --upgrade vecstack
• Upgrade vecstack WITHOUT upgrading dependencies:
  • pip install --upgrade --no-deps vecstack
• Upgrade directly from GitHub WITHOUT upgrading dependencies:
  • pip install --upgrade --no-deps https://github.com/vecxoz/vecstack/archive/master.zip
• Uninstall
  • pip uninstall vecstack

Usage. Functional API

from vecstack import stacking

# Get your data

# Initialize 1st level estimators
models = [LinearRegression(),
          Ridge(random_state=0)]

# Get your stacked features in a single line
S_train, S_test = stacking(models, X_train, y_train, X_test, regression=True, verbose=2)

# Use 2nd level estimator with stacked features

Usage. Scikit-learn API

from vecstack import StackingTransformer

# Get your data

# Initialize 1st level estimators
estimators = [('lr', LinearRegression()),
              ('ridge', Ridge(random_state=0))]
              
# Initialize StackingTransformer
stack = StackingTransformer(estimators, regression=True, verbose=2)

# Fit
stack = stack.fit(X_train, y_train)

# Get your stacked features
S_train = stack.transform(X_train)
S_test = stack.transform(X_test)

# Use 2nd level estimator with stacked features

Stacking FAQ

1. How can I report an issue? How can I ask a question about stacking or vecstack package?
2. How can I say thanks?
3. How to cite vecstack?
4. What is stacking?
5. What about stacking name?
6. Do I need stacking at all?
7. Can you explain stacking (stacked generalization) in 10 lines of code?
8. Why do I need complicated inner procedure for stacking?
9. I want to implement stacking (stacked generalization) from scratch. Can you help me?
10. What is OOF?
11. What are estimator, learner, model?
12. What is blending? How is it related to stacking?
13. How to optimize weights for weighted average?
14. What is better: weighted average for current level or additional level?
15. What is bagging? How is it related to stacking?
16. How many models should I use on a given stacking level?
17. How many stacking levels should I use?
18. How do I choose models for stacking?
19. I am trying hard but still can't beat my best single model with stacking. What is wrong?
20. What should I choose: functional API (stacking function) or Scikit-learn API (StackingTransformer)?
21. How do parameters of stacking function and StackingTransformer correspond?
22. Why Scikit-learn API was implemented as transformer and not predictor?
23. How to estimate stacking training time and number of models which will be built?
24. Which stacking variant should I use: 'A' ('oof_pred_bag') or 'B' ('oof_pred')?
25. How to choose number of folds?
26. [When I transform train set I see 'Train set w