Ploomber
The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️
Install / Use
/learn @ploomber/PloomberREADME
<p align="center"> <a href="https://ploomber.io/community">Join our community</a> | <a href="https://share.hsforms.com/1E7Qa_OpcRPi_MV-segFsaAe6c2g">Newsletter</a> | <a href="mailto:contact@ploomber.io">Contact us</a> | <a href="https://docs.ploomber.io/">Docs</a> | <a href="https://ploomber.io/blog/">Blog</a> | <a href="https://www.ploomber.io">Website</a> | <a href="https://www.youtube.com/channel/UCaIS5BMlmeNQE4-Gn0xTDXQ">YouTube</a> </p>[!TIP] Deploy AI apps for free on Ploomber Cloud!
Ploomber is the fastest way to build data pipelines ⚡️. Use your favorite editor (Jupyter, VSCode, PyCharm) to develop interactively and deploy ☁️ without code changes (Kubernetes, Airflow, AWS Batch, and SLURM). Do you have legacy notebooks? Refactor them into modular pipelines with a single command.
Installation
Compatible with Python 3.7 and higher.
Install with pip:
pip install ploomber
Or with conda:
conda install ploomber -c conda-forge
Getting started
Try the tutorial:
<p> <a href="https://github.com/ploomber/projects/blob/master/guides/first-pipeline/README.md"> </a> </p>Community
Main Features
⚡️ Get started quickly
A simple YAML API to get started quickly, a powerful Python API for total flexibility.
https://user-images.githubusercontent.com/989250/150660813-fc289c6c-0ed5-432d-b6df-063ce98c0093.mp4
⏱ Shorter development cycles
Automatically cache your pipeline’s previous results and only re-compute tasks that have changed since your last execution.
https://user-images.githubusercontent.com/989250/150660820-9a3a0abd-5904-492b-97ff-5494285dfebf.mp4
☁️ Deploy anywhere
Run as a shell script in a single machine or distributively in Kubernetes, Airflow, AWS Batch, or SLURM.
https://user-images.githubusercontent.com/989250/150660830-3f81c9a2-5392-49e5-976d-cb8a38441ecb.mp4
📙 Automated migration from legacy notebooks
Bring your old monolithic notebooks, and we’ll automatically convert them into maintainable, modular pipelines.
https://user-images.githubusercontent.com/989250/150660840-b0c12f85-504c-4233-8c3d-6724d291f1aa.mp4
I want to migrate my notebook.
Resources
- Documentation
- PyData Chicago talk (covers motivation and demo)
- Develop and deploy an ML pipeline in 30 minutes (EuroPython 2021)
- Guest blog post on the official Jupyter blog
- Examples (Machine Learning pipeline, ETL, among others)
- Blog
- Comparison with other tools
- More videos
About Ploomber
Ploomber is a big community of data enthusiasts pushing the boundaries of Data Science and Machine Learning tooling.
Whatever your skillset is, you can contribute to our mission. So whether you're a beginner or an experienced professional, you're welcome to join us on this journey!
