Scientific Python: guide, cookie, & sp-repo-review

Cookie

A copier/cookiecutter template for new Python projects based on the Scientific Python Developer Guide. What makes this different from other templates for Python packages?

• Lives with the [Scientific-Python Development Guide][]: Every decision is clearly documented and every tool described, and everything is kept in sync.
• Ten different backends to choose from for building packages.
• Optional VCS versioning for most backends.
• Selection for your preferred documentation engine (Sphinx, MkDocs, or Zensical).
• Template generation tested in GitHub Actions using nox.
• Supports generation with copier, cookiecutter, and cruft.
• Supports GitHub Actions if targeting a github.com url (the default), and adds experimental GitLab CI support otherwise.
• Includes several compiled backends using pybind11, with wheels produced for all platforms using cibuildwheel.
• Provides sp-repo-review to evaluate existing repos against the guidelines, with a WebAssembly version integrated with the guide. Checks are hyperlinked back to the guide.
• Follows PyPA best practices and regularly updated. Recent additions:
  • PEP 639: SPDX licenses
  • PEP 735: Dependency groups
• Uses uv for high performance CI and task running.

Be sure you have read the [Scientific-Python Development Guide][] first, and possibly used them on a project or two. This is not a minimal example or tutorial. It is a collection of useful tooling for starting a new project using cookiecutter, or for copying in individual files for an existing project (by hand, from {{cookiecutter.project_name}}/).

During generation you can select from the following backends for your package:

1. hatch: This uses hatchling, a modern builder with nice file inclusion, extendable via plugins, and good error messages. (Recommended for pure Python projects)
2. [uv][]: The uv_build backend is written in Rust and is integrated into' uv, meaning it can build without downloading anything extra and can even avoid running Python at all when building, making it the fastest backend for simple packages. No dynamic metadata support.
3. flit: A modern, lightweight PEP 621 build system for pure Python projects. Replaces setuptools, no MANIFEST.in, setup.py, or setup.cfg. Low learning curve. Easy to bootstrap into new distributions. Difficult to get the right files included, little dynamic metadata support.
4. pdm: A modern, less opinionated all-in-one solution to pure Python projects supporting standards. Replaces setuptools, venv/pipenv, pip, wheel, and twine. Supports PEP 621.
5. poetry: An all-in-one solution to pure Python projects. Replaces setuptools, venv/pipenv, pip, wheel, and twine. Higher learning curve, but is all-in-one. Makes some bad default assumptions for libraries.
6. [setuptools][]: The classic build system, but with the new standardized configuration.
7. pybind11: This is setuptools but with an C++ extension written in pybind11 and wheels generated by cibuildwheel.
8. [scikit-build][]: A scikit-build (CMake) project also using pybind11, using scikit-build-core. (Recommended for C++ projects)
9. meson-python: A Meson project also using pybind11. (No VCS versioning)
10. maturin: A PEP 621 builder for Rust binary extensions. (No VCS versioning) (Recommended for Rust projects)

Currently, the best choice is probably hatch for pure Python projects, and scikit-build (such as the scikit-build-core + pybind11 choice) for binary projects.

To use (copier version)

Install copier and copier-templates-extensions. Using [uv][], that's:

uv tool install --with copier-templates-extensions copier

Now, run copier to generate your project:

copier copy gh:scientific-python/cookie <pkg> --trust --vcs-ref=HEAD

(<pkg> is the path to put the new project. --vcs-ref=HEAD gets the current version instead of the last tag, matching cookiecutter's behavior. Note you can combine these two lines into one with uvx, just remember to pass --with before the program name in that case.)

You will get a nicer CLI experience with answer validation. You will also get a .copier-answers.yml file, which will allow you to perform updates in the future.

Note: Add --vcs-ref=HEAD to get the latest version instead of the last tagged version; HEAD always passes tests (and is what cookiecutter uses).

To use (cookiecutter version)

Install cookiecutter, ideally with brew install cookiecutter if you use brew, otherwise with uv tool install cookiecutter (or prepend uvx to the command below, and skip installation). Then run:

cookiecutter gh:scientific-python/cookie

If you are using cookiecutter 2.2.3+, you will get nice descriptions for the options like copier!

To use (cruft version)

You can also use cruft, which adds the ability update to cookiecutter projects. Install with uv tool install cruft (or prepend uvx to the command below, and skip installation). Then run:

cruft create https://github.com/scientific-python/cookie

Post generation

Check the key setup files, pyproject.toml, and possibly setup.cfg and setup.py (pybind11 example). Update README.md. Also update and add docs to docs/.

There are a few example dependencies and a minimum Python version of 3.10, feel free to change it to whatever you actually need/want. There is also a basic backports structure with a small typing example.

Contained components:

• GitHub Actions runs testing for the generation itself
  • Uses nox so cookie development can be checked locally
  • Uses uv for high performance CI
• GitHub actions deployment
  • C++ backends include cibuildwheel for wheel builds
  • Uses PyPI trusted publisher deployment
• A GitHub release configuration that skips common bot commits when auto-generating release notes
• Dependabot keeps actions up to date periodically, through useful pull requests
• A contributing guide
• Formatting handled by pre-commit or prek
  • No reason not to be strict on a new project; remove what you don't want.
  • Includes MyPy - static typing
  • Includes Ruff - standard formatting, linting and autofixes
    • Replaces Flake8, isort, pyupgrade, yesqa, pycln, and dozens of plugins
  • Includes spell checking
• One of several popular licenses
• A pylint nox target can be used to run pylint, which integrated GHA annotations
• A ReadTheDocs-ready docs/ folder and docs dependency-group
• A tests/ folder and pytest test dependency-group
• A dev group for uv run integration
• A noxfile is included with a few common targets
• A README
• Code coverage reporting with automatic uploads to Codecov after tests run

For developers:

You can test locally with nox:

# See all commands
nox -l

# Run a specific check
nox -s "lint(scikit-build)"

# Run a noxfile command on the project noxfile
nox -s "nox(hatch)" -- docs

If you don't have nox locally, you can use [uv][], such as uvx nox instead.

Other similar projects

Hypermodern-Python is another project worth checking out with many similarities, like great documentation for each feature and many of the same tools used. It has a slightly different set of features, and has a stronger focus on GitHub Actions - most our guide could be adapted to a different CI system fairly easily if you don't want to use GHA. It also forces the use of Poetry (instead of having a backend selection), and doesn't support compiled projects. It currently dumps all development dependencies into a shared environment, causing long solve times and high chance of conflicts. It also does not use pre-commit the way it was intended to be used. It also has quite a bit of custom code.

History

A lot of the guide, cookiecutter, and repo-review started out as part of [Scikit-HEP][]. These projects were merged, generalized, and combined with the NSLS-II guide during the 2023 Scientific-Python Developers Summit.

<!-- prettier-ignore-start -->