Lithops is a Python multi-cloud distributed computing framework that lets you run unmodified Python code at massive scale across cloud, HPC, and on-premise platforms. It supports major cloud providers and Kubernetes platforms, running your code transparently without requiring you to manage deployment or infrastructure.

Lithops is ideal for highly parallel workloads—such as Monte Carlo simulations, machine learning, metabolomics, or geospatial analytics—and lets you tailor execution to your priorities: you can optimize for performance using AWS Lambda to launch hundreds of functions in milliseconds, or reduce costs by running the same code on AWS Batch with Spot Instances.

Installation

Install Lithops from the PyPi repository:
```
pip install lithops
```
Execute a Hello World function:
```
lithops hello
```

Configuration

Lithops provides an extensible backend architecture (compute, storage) designed to work with various cloud providers and on-premise platforms. You can write your code in Python and run it unmodified across major cloud providers and Kubernetes environments.

Follow these instructions to configure your compute and storage backends

High-level API

Lithops is shipped with 2 different high-level Compute APIs, and 2 high-level Storage APIs

<div align="center"> <table> <tr> <th> <img width="50%" height="1px"> <p><small><a href="docs/api_futures.md">Futures API</a></small></p> </th> <th> <img width="50%" height="1px"> <p><small><a href="docs/source/api_multiprocessing.rst">Multiprocessing API</a></small></p> </th> </tr> <tr> <td>

from lithops import FunctionExecutor

def double(i):
    return i * 2

with FunctionExecutor() as fexec:
    f = fexec.map(double, [1, 2, 3, 4])
    print(f.result())

</td> <td>

from lithops.multiprocessing import Pool

def double(i):
    return i * 2

with Pool() as pool:
    result = pool.map(double, [1, 2, 3, 4])
    print(result)

</td> </tr> </table> <table> <tr> <th> <img width="50%" height="1px"> <p><small><a href="docs/api_storage.md">Storage API</a></small></p> </th> <th> <img width="50%" height="1px"> <p><small><a href="docs/source/api_storage_os.rst">Storage OS API</a></small></p> </th> </tr> <tr> <td>

from lithops import Storage

if __name__ == "__main__":
    st = Storage()
    st.put_object(bucket='mybucket',
                  key='test.txt',
                  body='Hello World')

    print(st.get_object(bucket='lithops',
                        key='test.txt'))

</td> <td>

from lithops.storage.cloud_proxy import os

if __name__ == "__main__":
    filepath = 'bar/foo.txt'
    with os.open(filepath, 'w') as f:
        f.write('Hello world!')

    dirname = os.path.dirname(filepath)
    print(os.listdir(dirname))
    os.remove(filepath)

</td> </tr> </table> </div>

You can find more usage examples in the examples folder.

Documentation

For documentation on using Lithops, see latest release documentation

If you are interested in contributing, see CONTRIBUTING.md.

Additional resources

Blogs and Talks

How to run Lithops over EC2 VMs using the new K8s backend
Simplify the developer experience with OpenShift for Big Data processing by using Lithops framework
Speed-up your Python applications using Lithops and Serverless Cloud resources
Lithops, a Multi-cloud Serverless Programming Framework
CNCF Webinar - Toward Hybrid Cloud Serverless Transparency with Lithops Framework
Your easy move to serverless computing and radically simplified data processing Strata Data Conference, NY 2019. See video of Lithops usage here and the example of Monte Carlo here

Papers

Serverful Functions: Leveraging Servers in Complex Serverless Workflows - ACM Middleware Industrial Track 2024
Transparent serverless execution of Python multiprocessing applications - Elsevier Future Generation Computer Systems 2023
Outsourcing Data Processing Jobs with Lithops - IEEE Transactions on Cloud Computing 2022
Towards Multicloud Access Transparency in Serverless Computing - IEEE Software 2021
Primula: a Practical Shuffle/Sort Operator for Serverless Computing - ACM/IFIP International Middleware Conference 2020. See presentation here
Bringing scaling transparency to Proteomics applications with serverless computing - 6th International Workshop on Serverless Computing (WoSC6) 2020. See presentation here
Serverless data analytics in the IBM Cloud - ACM/IFIP International Middleware Conference 2018

Acknowledgements

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825184 (CloudButton).

Lithops

Install / Use

README