pydis

pydis is an experiment to disprove some of the falsehoods about performance and optimisation regarding software and interpreted languages in particular.

Below you will find a Redis clone, pydis, written in ~250 lines of idiomatic Python code, providing a subset of redis' functionality for which there are official benchmarks.

Briefly, pydis is ~50% as fast as Redis measured in number operations per second.

P.S. This is not a criticism of Redis, which is a brilliant project and a system-level software that powers thousands of infrastructures. It just happened to be one of the fastest software I could imagine and clone the same day.

Disclaimer

I have used the following libraries written in C for performance:

uvloop

uvloop is a fast, drop-in replacement of the built-in asyncio event loop. uvloop is implemented in Cython and uses libuv under the hood. >
hiredis

Python extension that wraps protocol parsing code in hiredis.

Discussion

Hacker News
1. 2020-11-15 -- Pydis – Redis clone in 250 lines of Python, for performance comparison
2. 2019-03-02 -- Pydis: Redis clone in Python 3 to make points about performance
Reddit
1. 2019-03-01 -- pydis - A redis clone in Python 3 to disprove some falsehoods about performance

Results

redis-benchmark -q -t set,get,incr,lpush,rpush,lpop,rpop,sadd,hset,spop,lrange,mset -n 100000 -P 5

100,000 requests in total per command.
Requests are pipelined in groups of 5.

Benchmark | pydis | redis | Ratio --- | ---: | ---: | --- SET | 271,947 | 467,361 | 0.582 GET | 274,283 | 467,237 | 0.587 INCR | 213,409 | 478,669 | 0.446 LPUSH | 216,082 | 381,033 | 0.567 RPUSH | 231,143 | 399,238 | 0.579 LPOP | 248,527 | 384,332 | 0.647 RPOP | 241,144 | 429,971 | 0.561 SADD | 219,475 | 434,257 | 0.505 HSET | 220,178 | 377,637 | 0.583 SPOP | 288,068 | 477,705 | 0.603 LRANGE (100) | 26,170 | 96,254 | 0.272 LRANGE (300) | 9,163 | 24,768 | 0.370 LRANGE (500) | 5,771 | 19,351 | 0.298 LRANGE (600) | 4,705 | 13,869 | 0.339 MSET | 125,215 | 195,121 | 0.642

Host System

Ubuntu 20.04
Python 3.8.5 (GCC 9.3.0)
Redis 5.0.7 malloc=jemalloc-5.2.1 bits=64 build=636cde3b5c7a3923

Contributions

Contributions are very welcome, given that they fall into one of the following categories:

Those that improve the performance.
- The aim of this exercise is to prove that interpreted languages can be just as fast as C. So whilst using a faster parser in C with Python bindings is okay, rewriting pydis in Cython is not.
- I will accept "minor" deviations from idioms only if the performance gains are worth it; stick to idiomatic Python otherwise!
Those to achieve feature parity with Redis for which there are official benchmarks.
- We are not trying to develop a full-featured Redis clone here so please do not implement commands for which there are no official benchmarks.
Those that fix formatting etc.
- Please do not invent your own style, use PEP 8.

Pydis

Install / Use

README