.. image:: ./documentation/assets/github-banner.png

############################################# kawipiko -- blazingly fast static HTTP server #############################################

.. highlights::

Table of contents:

* `Manual <#manual>`__ and `Examples <#examples>`__
* `Installation <#installation>`__ and `FAQ <#faq>`__
* `Features <#features>`__ and `Benchmarks <#benchmarks>`__
* `About <#about>`__, `Copyright and licensing <#notice-copyright-and-licensing>`__, `SBOM <#sbom-software-bill-of-materials>`__, and `References <#references>`__
* `chat on Discord <https://discord.gg/NH7V7NKjNu>`__, `discuss on GitHub <https://github.com/volution/kawipiko/discussions/categories/discussions>`__, or `email author <mailto:ciprian.craciun@gmail.com>`__

About

kawipiko is a lightweight static HTTP server written in Go; focused on serving static content as fast and efficient as possible, with the lowest latency, and with the lowest resource consumption (either CPU, RAM, IO); supporting both HTTP/1 (with or without TLS), HTTP/2 and HTTP/3 (over QUIC); available as a single statically linked executable without any other dependencies.

Want to see kawipiko in action?

kawipiko-examples.volution.ro <https://kawipiko-examples.volution.ro/>__ -- here are a few demos that best fit kawipiko; /documentation represents static sites that compress well, meanwhile /carina-nebula, /openstreetmap and /fontawesome represents static sites that generate lots of small requests; (please note that this instance is behind CloudFlare, however it was instructed not to cache anything on the edge;) (the server is currently hosted over a residential fiber connection;)
notes.volution.ro <https://notes.volution.ro/>__ -- my personal notes / blog site, previously hosted at Netlify; (the server is currently hosted over a residential fiber connection;)

However, simple doesn't imply dumb or limited, instead it implies efficient through the removal of superfluous features, thus being inline with UNIX's old philosophy of "do one thing and do it well <https://en.wikipedia.org/wiki/Unix_philosophy#Do_One_Thing_and_Do_It_Well>__". Therefore, it supports only GET requests, and does not provide features like dynamic content generation, authentication, reverse proxying, etc.; meanwhile still providing compression (gzip, zopfli, or brotli), plus HTML-CSS-JS minifying (TODO), without affecting its performance (due to its unique architecture as described below).

What kawipiko does provide is something very unique, that no other HTTP server offers: the static content is served from a CDB file <#why-cdb>__ with almost no latency (as compared to classical static servers that still have to pass through the OS via the open-read-close syscalls). Moreover, as noted earlier, the static content can still be compressed or minified ahead of time, thus reducing not only CPU but also bandwidth and latency.

CDB files <#why-cdb>__ are binary database files that provide efficient read-only key-value lookup tables, initially used in some DNS and SMTP servers, mainly for their low overhead lookup operations, zero locking in multi-threaded / multi-process scenarios, and "atomic" multi-record updates. This also makes them suitable for low-latency static content serving over HTTP, which is what this project provides.

For those familiar with Netlify (or competitors like CloudFlare Pages, GitHub Pages, etc.), kawipiko is a host-it-yourself alternative featuring:

self-contained deployment with simple configuration; (i.e. just fetch the executable <#installation>__ and use the proper flags <#kawipiko-server>__;)
low and constant resource consumption (both in terms of CPU and RAM); (i.e. you won't have surprises when under load;)
(hopefully) extremely secure; (i.e. it doesn't launch processes, it doesn't connect to other services or databases, it doesn't open any files, etc.; basically you can easily chroot it, or containerize it as is in fashion these days;)
highly portable, supporting at least Linux (the main development, testing and deployment platform), FreeBSD, OpenBSD, and OSX;

For a complete list of features please consult the features section <#features>. Unfortunately, there are also some tradeoffs as described in the limitations section <#limitations> (although none are critical).

With regard to performance, as described in the benchmarks section <#benchmarks>__, kawipiko is at least on-par with NGinx, sustaining over 100K requests / second with 0.25ms latency for 99% of the requests even on my 6 years old laptop. However the main advantage over NGinx is not raw performance, but deployment and configuration simplicity, plus efficient management and storage of large collections of many small files.

In relation to kawipiko I've also published a few articles on my own site:

Securing my static site server with seccomp <https://notes.volution.ro/v1/2022/09/remarks/794866f7/>__ -- describes how I have employed Linux's seccomp syscall filtering to improve the security by reducing the attack surface;
Static site hosting hurdres <https://notes.volution.ro/v1/2022/09/notes/b08118d8/>__ -- describes the reasoning behind implementing this simple HTTP server;
A visual comparison between HTTP/1.1 and HTTP/2 <https://notes.volution.ro/v1/2019/08/notes/e8700e9a/>__ -- a few experiments in how Firefox and Chromium handle lots of resource loading over HTTP/1.1 vs HTTP/2; (the server used for the expermients was kawipiko;)

Manual

.. contents:: :local: :backlinks: none

Workflow

The project provides the following executables (statically linked, without any other dependencies):

kawipiko-server -- which serves the static content from the CDB archive either via HTTP (with or without TLS), HTTP/2 or HTTP/3 (over QUIC);
kawipiko-archiver -- which creates the CDB archive from a source folder holding the static content, optionally compressing and minifying files;
kawipiko -- an all-in-one executable that bundles all functionality in one executable; (i.e. kawipiko server ... or kawipiko archiver ...);

Unlike most (if not all) other servers out-there, in which you just point your web server to the folder holding the static website content root, kawipiko takes a radically different approach: in order to serve the static content, one has to first archive the content into the CDB archive through kawipiko-archiver, and then one can serve it from the CDB archive through kawipiko-server.

This two step phase also presents a few opportunities:

one can decouple the "building", "testing", and "publishing" phases of a static website, by using a similar CI/CD pipeline as done for other software projects;
one can instantaneously rollback to a previous version if the newly published one has issues;
one can apply extreme compression (e.g. zopfli or brotli), to trade CPU during deployment vs latency and bandwidth at runtime.

kawipiko-server

See the dedicated manual <./documentation/manuals/server.rst>__.

This document is also available in plain text <./documentation/manuals/server.txt>, or as a man page <./documentation/manuals/server.1.man>.

kawipiko-archiver

See the dedicated manual <./documentation/manuals/archiver.rst>__.

This document is also available in plain text <./documentation/manuals/archiver.txt>, or as a man page <./documentation/manuals/archiver.1.man>.

Examples

fetch and extract the Python 3.10 documentation HTML archive: ::

curl
-s -S -f
-o ./python-3.10.1-docs-html.tar.bz2
https://docs.python.org/3/archives/python-3.10.1-docs-html.tar.bz2 \

tar
-x -j -v
-f ./python-3.10.1-docs-html.tar.bz2 \
create the CDB archive (without any compression): ::

kawipiko-archiver
--archive ./python-3.10.1-docs-html-nocomp.cdb
--sources ./python-3.10.1-docs-html
--debug \
create the CDB archive (with gzip compression): ::

kawipiko-archiver
--archive ./python-3.10.1-docs-html-gzip.cdb
--sources ./python-3.10.1-docs-html
--compress gzip
--debug \
create the CDB archive (with zopfli compression): ::

kawipiko-archiver
--archive ./python-3.10.1-docs-html-zopfli.cdb
--sources ./python-3.10.1-docs-html
--compress zopfli
--debug \
create the CDB archive (with brotli compression): ::

kawipiko-archiver
--archive ./python-3.10.1-docs-html-brotli.cdb
--sources ./python-3.10.1-docs-html
--compress brotli
--debug \
serve the CDB archive (with gzip compression): ::

kawipiko-server
--bind 127.0.0.1:8080
--archive ./python-3.10.1-docs-html-gzip.cdb
--archive-mmap
--archive-preload
--debug \
compare sources and archive sizes: ::

du
-h -s

./python-3.10.1-docs-html-nocomp.cdb
./python-3.10.1-docs-html-gzip.cdb
./python-3.10.1-docs-html-zopfli.cdb
./python-3.10.1-docs-html-brotli.cdb

./python-3.10.1-docs-html
./python-3.10.1-docs-html.tar.bz2 \

45M ./python-3.10.1-docs-html-nocomp.cdb 9.7M ./python-3.10.1-docs-html-gzip.cdb ??? ./python-3.10.1-docs-html-zopfli.cdb 7.9M ./python-3.10.1-docs-html-brotli.cdb

46M ./python-3

Kawipiko

Install / Use

README