Knapsack
Knapsack splits tests evenly across parallel CI nodes to run fast CI build and save you time.
Install / Use
/learn @KnapsackPro/KnapsackREADME
<p align="center"> <a href="https://knapsackpro.com?utm_source=github&utm_medium=readme&utm_campaign=knapsack_gem_archived&utm_content=hero_logo"> <img alt="Knapsack" src="./.github/assets/knapsack-diamonds.png" width="300" height="300" style="max-width: 100%;" /> </a> </p> <h3 align="center">Speed up your tests</h3> <p align="center">Run your 1-hour test suite in 2 minutes with optimal parallelisation on your existing CI infrastructure</p>[!WARNING] Knapsack is archived. But Knapsack Pro is available.
Knapsack Pro comes with a free plan for OSS projects and discounts on paid plans for people coming from Knapsack (see how to migrate in 10 minutes).
This repository remains available to fork and the gem hosted on RubyGems, so your existing setup won't be affected.
<div align="center"> <a href="https://rubygems.org/gems/knapsack"> <img alt="Gem Version" src="https://badge.fury.io/rb/knapsack.svg" /> </a> </div> <br /> <br />
Knapsack wraps your current test runner and works with your existing CI infrastructure to split tests optimally.
It comes in two flavors, knapsack and knapsack_pro:
| | knapsack | knapsack_pro |
| ------------------------------- | ---------- | --------------------------------------- |
| Free | ✅ | ✅ OSS projects |
| Static split | ✅ | ✅ |
| Dynamic split | ❌ | ✅ |
| Split by test examples | ❌ | ✅ |
| Graphs, metrics, and stats | ❌ | ✅ |
| Programming languages | 🤞 (Ruby) | ✅ (Ruby, Cypress, Jest, SDK/API) |
| CI providers | 🤞 Limited | ✅ (All) |
| Heroku add-on | ❌ | ✅ |
| Automated execution time recording | ❌ | ✅ |
| Test split based on most recent execution times | ❌ | ✅ |
| Support for spot/preemptible CI nodes | ❌ | ✅ |
| Additional features | ❌ | 🤘 (Overview) |
| | Install | Install |
Migrate from knapsack to knapsack_pro
If you already use knapsack and want to give knapsack_pro a try, here's how to migrate in 10 minutes.
knapsack
Knapsack generates a test time execution report and uses it for future test runs.
The knapsack gem supports:
Without Knapsack - bad test suite split
With Knapsack - better test suite split
Requirements
>= Ruby 2.1.0
<!-- START doctoc generated TOC please keep comment here to allow auto update --> <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
- Update
- Installation
- Usage
- Set up your CI server
- FAQ
- What does time offset warning mean?
- How to generate the Knapsack report?
- What does "leftover specs" mean?
- Why are there "leftover specs" after I generate a new report?
- How can I run tests from multiple directories?
- How to update the existing Knapsack report for a few test files?
- How to run tests for particular CI node in your development environment
- How can I change the log level?
- Gem tests
- Acknowledgements
- Mentions
Update
Please check CHANGELOG.md before updating the gem. Knapsack follows semantic versioning.
Installation
Add these lines to your application's Gemfile:
group :test, :development do
gem 'knapsack'
end
And then execute:
bundle
Add this line at the bottom of Rakefile:
Knapsack.load_tasks if defined?(Knapsack)
Usage
Here's an example of a Rails app with Knapsack.
https://github.com/KnapsackPro/rails-app-with-knapsack
Step for RSpec
Add at the beginning of your spec_helper.rb:
require 'knapsack'
# CUSTOM_CONFIG_GOES_HERE
Knapsack::Adapters::RSpecAdapter.bind
Step for Cucumber
Create features/support/knapsack.rb:
require 'knapsack'
# CUSTOM_CONFIG_GOES_HERE
Knapsack::Adapters::CucumberAdapter.bind
Step for Minitest
Add the Knapsack code after you load the app environment in test/test_helper.rb:
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
require 'knapsack'
# CUSTOM_CONFIG_GOES_HERE
knapsack_adapter = Knapsack::Adapters::MinitestAdapter.bind
knapsack_adapter.set_test_helper_path(__FILE__)
Step for Spinach
Create features/support/env.rb:
require 'knapsack'
# CUSTOM_CONFIG_GOES_HERE
Knapsack::Adapters::SpinachAdapter.bind
Custom configuration
You can change the default Knapsack configuration for RSpec, Cucumber, Minitest, or Spinach tests.
Here are some examples (that you can insert in CUSTOM_CONFIG_GOES_HERE):
Knapsack.tracker.config({
enable_time_offset_warning: true,
time_offset_in_seconds: 30
})
Knapsack.report.config({
test_file_pattern: 'spec/**{,/*/**}/*_spec.rb', # default value based on adapter
report_path: 'knapsack_custom_report.json'
})
# You can use your logger:
require 'logger'
Knapsack.logger = Logger.new(STDOUT)
Knapsack.logger.level = Logger::INFO
Common step
Generate the time execution report for your test files. Run the command below on one of your CI nodes:
# Step for RSpec:
KNAPSACK_GENERATE_REPORT=true bundle exec rspec spec
# Step for Cucumber:
KNAPSACK_GENERATE_REPORT=true bundle exec cucumber features
# Step for Minitest:
KNAPSACK_GENERATE_REPORT=true bundle exec rake test
# If you use Rails 5.0.x then run this instead:
KNAPSACK_GENERATE_REPORT=true bundle exec rake test
# If you use Rails >= 5.1's SystemTest, run both unit and system tests:
KNAPSACK_GENERATE_REPORT=true bundle exec rake test test:system
# Step for Spinach:
KNAPSACK_GENERATE_REPORT=true bundle exec spinach
Commit the generated report knapsack_rspec_report.json, knapsack_cucumber_report.json, knapsack_minitest_report.json or knapsack_spinach_report.json into your repository.
This report should be updated after you add a lot of new slow tests or you change existing ones, which causes a big time execution difference between CI nodes.
You will get a time offset warning at the end of the RSpec/Cucumber/Minitest run, which reminds you when it’s a good time to regenerate the Knapsack report.
KNAPSACK_GENERATE_REPORT is truthy with "true" or 0. All other values are falsy, though "false" and 1 are semantically preferrable.
Adding or removing tests
There is no need to regenerate the report every time you add/remove test files.
If you
