pgzx - Create Postgres Extensions with Zig!

pgzx is a library for developing PostgreSQL extensions written in Zig. It provides a set of utilities (e.g. error handling, memory allocators, wrappers) as well as a development environment that simplifies integrating with the Postgres code base.

Why Zig?

Zig is a small and simple language that aims to be a "modern C" and make system-level code bases easier to maintain. It provides safe memory management, compilation time code execution (comptime), and a rich standard library.

Zig can interact with C code quite naturally: it supports the C ABI, can work with C pointers and types directly, it can import header files and even translate C code to Zig code. Thanks to this interoperability, a Postgres extension written in Zig can, theoretically, accomplish anything that a C extension can. This means you get full power AND a modern language and standard library to write your extension.

While in theory you can write any extension in Zig that you could in C, in practice you will need to make sense of a lot of Postgres internals in order to know how to correctly use them from Zig. Also, Postgres makes extensive use of macros, and not all of them can be translated automatically. This is where pgzx comes in: it provides a set of Zig modules that make the development of Postgres Extensions in Zig much simpler.

Examples

The following sample extensions (ordered from simple to complex) show how to use pgzx:

| Extension | Description | |--------------------------------------------|-------------| | char_count_zig | Adds a function that counts how many times a particular character shows up in a string. Shows how to register a function and how to interpret the parameters. | | pghostname_zig | Adds a function that returns the database server's host name. | | pg_audit_zig | Inspired by the pgaudit C extension, this one registers callbacks to multiple hooks and uses more advanced error handling and memory allocation patterns. |

Docs

The reference documentation is available at here.

We recommend checking the examples in the section above to understand how to use pgzx. The next sections contain a high-level walkthrough of the most important utilities and how they relate to the Postgres internals.

Getting Started

This project uses Nix flakes to manage build dependencies and provide a development shell. We provide a template for you to initialize a new Zig based Postgres extension project which allows you to reuse some of the utilities we're using.

Before getting started we would recommend you familiarize yourself with the projects setup first. To do so, please start with the Contributing section.

We will create a new project folder for our new extension and initialize the folder using the projects template:

$ mkdir my_extension
$ cd my_extension
$ nix flake init -t github:xataio/pgzx

This step will create a working extension named 'my_extension'. The extension exports a hello world function named hello().

The templates README.md file already contains instructions on how to enter the development shell, build, and test the extension. You can follow the instructions and verify that your setup is functioning. Do not forget to use pgstop before quitting the development shell.

The development shell declares a few environment variables used by the project (see devshell.nix):

PRJ_ROOT: folder of the current project. If not set some shell scripts will ask git to find the projects folder. Some scripts use this environment variable to ensure that you can run the script from within any folder within your project.
PG_HOME: Installation path of your postgres instance. When building postgres from scratch this matches the path prefix used by make install. When using the development shell we will relocate/build the postgres extension into the ./out folder and create a symlink ./out/default to the local version. If you plan to build and install the extension with another PostgreSQL installation set PG_HOME=$(dirname $(pg_config --bindir)).

Next we want to rename the project to match our extension name. To do so, update the file names in the extension folder, and replace my_extension with your project name in the README.md, build.zig, build.zig.zon, and extensions SQL file.

Logging and error handling

Postgres error reporting functions are used to report errors and log messages. They have typical logging functionality like log levels and formatting, but also Postgres specific functionality, like error reports that can be thrown and caught like exceptions. pgzx provides a wrapper around these functions that makes it easier to use from Zig.

Simple logging can be done with functions like [Log][docs_Log], [Info][docs_Info], [Notice][docs_Notice], [Warning][docs_Warning], for example:

    elog.Info(@src(), "input_text: {s}\n", .{input_text});

Note the @src() built-in which provides the file location. This will be stored in the error report.

To report errors during execution, use the [Error][docs_Error] or [ErrorThrow][docs_ErrorThrow] functions. The latter will throw an error report, which can be caught by the Postgres error handling system (explained below). Example with Error:

    if (target_char.len > 1) {
        return elog.Error(@src(), "Target char is more than one byte", .{});
    }

The elog module also exports functions that resemble the C API including functions like ereport, errcode, or errmsg.

If you browse through the Postgres source code, you'll see the PG_TRY / PG_CATCH / PG_FINALLY macros used as a form of "exception handling" in C, catching errors raised by the ereport family of functions. These macros make use of long jumps (i.e. jumps across function boundaries) to the "catch/finally" destination. This means we need to be careful when calling Postgres functions from Zig. For example, if the called C function raises an ereport error, the long jump might skip the Zig code that would have cleaned up resources (e.g. errdefer).

pgzx offers an alternative Zig implementation for the PG_TRY family of macros. This typically looks in code something like this:

    var errctx = pgzx.err.Context.init();
    defer errctx.deinit();
    if (errctx.pg_try()) {
        // zig code that calls several Postgres C functions.
    } else {
        return errctx.errorValue();
    }

The above code pattern makes sure that we catch any errors raised by Postgres functions and return them as Zig errors. This way, we make sure that all the defer and errdefer code in the caller(s) are executed as expected. For more details, see the documentation for the [pgzx.err.Context][docs_Context] struct.

The above code pattern is implemented in a [wrap][docs_wrap] convenience function which takes a function and its arguments, and executes it in a block like the above. For example:

    try pgzx.err.wrap(myFunction, .{arg1, arg2});

Memory context allocators

Postgres uses a memory context system to manage memory. Memory allocated in a context can be freed all at once (for example, when a query execution is finished), which simplifies memory management significantly, because you only need to track contexts, not individual allocations. Contexts are also hierarchical, so you can create a context that is a child of another context, and when the parent context is freed, all children are freed as well.

pgzx offers custom wrapper Zig allocators that use Postgres' memory context system. The [pgzx.mem.createAllocSetContext][docs_createAllocSetContext] function creates an [pgzx.mem.MemoryContextAllocator][docs_MemoryContextAllocator] that you can use as a Zig allocator. For example:

    var memctx = try pgzx.mem.createAllocSetContext("zig_context", .{ .parent = pg.CurrentMemoryContext });
    const allocator = memctx.allocator();

In the above, note the use of pg.CurrentMemoryContext as the parent context. This is the context of the current query execution, and it will be freed when the query is finished. This means that the memory allocated with allocator will be freed at the same time.

It's also possible to register a callback for when the memory context is destroyed or reset. This is useful to free or close resources that are tied to the context (e.g. sockets). pgzx provides an utility to register a callback:

    try memctx.registerAllocResetCallback(
        queryDesc.*.estate.*.es_query_cxt,
        pgaudit_zig_MemoryContextCallback,
    );

Function manager

pgzx has utilities for registering functions, written in Zi

Pgzx

Install / Use

README