Pbandk

Pbandk is a Kotlin Multiplatform code generator and runtime for Protocol Buffers.

NOTE: This is the documentation for the version of pbandk currently in development. Documentation for the latest stable version is available at https://github.com/streem/pbandk/blob/v0.16.0/README.md.

Summary

• Project Goals
• Features
• Status
• Generated Code Sample
• Usage
• Generated Code
• Building
• Releasing
• Credits

Project Goals

• Generate idiomatic Kotlin code from protobuf definitions
• Support all platforms that are supported by Kotlin Multiplatform (JVM, Android, iOS, JS, WasmJs, WasmWasi, Linux, Windows, etc.)
• Provide an API (both pbandk's public API and generated code) that is easy to use from other Kotlin Multiplatform code and follows Kotlin conventions
  • NOTE: We make a best effort to provide an API that is also easy to use directly from platform-specific code (Swift, Javascript, Typescript, Java). However, Kotlin Multiplatform interoperability with other languages still has many limitations that make this difficult to implement in practice. Pbandk's main focus is on supporting Kotlin developers. But issues and pull requests aimed at improving the pbandk API exposed to other languages are still welcome. Also see below for platform-specific notes.
• Full protobuf conformance test compatibility
• Binary and source backwards compatibility for using pbandk and pbandk-generated code from Kotlin
  • Pbandk libraries follow standard semantic versioning conventions (code compiled against an older version of pbandk should be compatible with newer pbandk versions at runtime)
  • Code generated by an older version of pbandk should be compatible with newer pbandk versions at runtime
  • Backwards compatible changes to the proto definitions should result in binary and source compatible changes in the generated code
• Until version 1.0.0, the focus is on protobuf feature parity and correctness. Performance, code size, memory impact, etc. are of course always considered as well, and contributions in these areas are welcome. But they will not be a primary focus until after 1.0.0.

Features

• Clean data class generation
• Works on JVM, Android, and Native (iOS, macOS, Linux, Windows, etc.), with experimental support for JS and Wasm
• Support for proto2 and proto3 syntaxes (support for Protobuf Editions is planned)
• JSON serialization/deserialization following the proto3 JSON spec (see https://github.com/streem/pbandk/issues/72 for some corner cases and Well-Known Types that are not handled yet)
• Oneof's are properly handled as sealed classes
• Specialized support to handle wrappers from the well-known types (e.g. StringValue, BoolValue) as nullable primitives (String?, Boolean?, etc.)
• Support for custom service/gRPC code generator
• Support for custom options
• Support for the deprecated protobuf option when generating Kotlin code

Platform-Specific Notes

• Kotlin/JS (js)
  • Support for Kotlin/JS in pbandk is still experimental. Using pbandk from other Kotlin Multiplatform code on Kotlin/JS is mostly stable. Using pbandk and pbandk-generated code directly from Javascript/Typescript code is not well supported yet because of limitations in Kotlin's @JsExport functionality.
• Kotlin/Wasm (wasmJs)
  • Support for Kotlin/Wasm in pbandk is still experimental. There is no support for using pbandk APIs and pbandk-generated code directly from Javascript code (or other Wasm host languages). The APIs can only be called from other Kotlin code that is being compiled for Kotlin/Wasm. Kotlin/Wasm doesn't currently support using classes or complex types as arguments for Kotlin functions that are exposed to the Wasm host environment. This makes it infeasible to expose pbandk APIs directly outside of Kotlin.
• Kotlin/Native - Windows (mingwX64)
  • Windows is supported on a best-effort basis. The pbandk runtime is available on the mingwX64 platform. The official protobuf project does not yet support running the protobuf conformance tests on Windows, so we cannot fully validate pbandk's correctness when run on Windows. However, almost all of the mingwX64 pbandk code is shared with the other native platforms (Linux, macOS), so the passing conformance tests on those platforms still provide a pretty good guarantee of pbandk's correctness on mingwX64.

Not Yet Implemented

• Specialized support for some of the less-common "well known types" (e.g. FieldMask)
• Access to the protobuf descriptor from generated code
• Code comments on generated code

Read below for more information and see the examples.

Status

This project is currently in beta. It has the core set of protobuf features implemented and is being used in production. But it is still under active development and new versions might introduce backwards-incompatible changes to support new features or to improve the library's usability in Kotlin. Pull requests are welcome for any of the "Not Yet Implemented" features above.

This project follows semantic versioning. After v1.0.0 is released, future versions will preserve backwards compatibility.

The project currently has a single maintainer (@garyp) working on it in his spare time. Contributors who would like to become additional maintainers are highly welcome. Your contributions don't have to be in the form of code and could also be documentation improvements, issue triage, community outreach, etc.

Support

For support or discussion relating to pbandk, please use the GitHub Discussions on this project. You can also find some of us in the #pbandk channel of the Kotlin Slack instance. Please drop in and say hi!

Generated Code Sample

For the following addressbook.proto file:

syntax = "proto3";
package tutorial;

import "google/protobuf/timestamp.proto";

message Person {
    string name = 1;
    int32 id = 2;
    string email = 3;

    enum PhoneType {
        MOBILE = 0;
        HOME = 1;
        WORK = 2;
    }

    message PhoneNumber {
        string number = 1;
        PhoneType type = 2;
    }

    repeated PhoneNumber phones = 4;

    google.protobuf.Timestamp last_updated = 5;
}

message AddressBook {
    repeated Person people = 1;
}

The following file will be generated at tutorial/addressbook.kt:

@file:OptIn(pbandk.PublicForGeneratedCode::class)

package tutorial

@pbandk.Export
public data class Person(
    val name: String = "",
    val id: Int = 0,
    val email: String = "",
    val phones: List<tutorial.Person.PhoneNumber> = emptyList(),
    val lastUpdated: pbandk.wkt.Timestamp? = null,
    override val unknownFields: Map<Int, pbandk.UnknownField> = emptyMap()
) : pbandk.Message {
    override operator fun plus(other: pbandk.Message?): tutorial.Person = protoMergeImpl(other)
    override val descriptor: pbandk.MessageDescriptor<tutorial.Person> get() = Companion.descriptor
    override val protoSize: Int by lazy { super.protoSize }
    public companion object : pbandk.Message.Companion<tutorial.Person> {
        public val defaultInstance: tutorial.Person by lazy { tutorial.Person() }
        override fun decodeWith(u: pbandk.MessageDecoder): tutorial.Person = tutorial.Person.decodeWithImpl(u)

        override val descriptors: pbandk.MessageDescriptor<tutorial.Person> = pbandk.MessageDescriptor(
            fullName = "tutorial.Person",
            messageClass = tutorial.Person::class,
            messageCompanion = this,
            fields = buildList(5) {
                add(
                    pbandk.FieldDescriptor(
                        messageDescriptor = this@Companion::descriptor,
                        name = "name",
                        number = 1,
                        type = pbandk.FieldDescriptor.Type.Primitive.String(),
                        jsonName = "name",
                        value = tutorial.Person::name
                    )
                )
                add(
                    pbandk.FieldDescriptor(
                        messageDescriptor = this@Companion::descriptor,
                        name = "id",
                        number = 2,
                        type = pbandk.FieldDescriptor.Type.Primitive.Int32(),
                        jsonName = "id",
                        value = tutorial.Person::id
                    )
                )
                add(
                    pbandk.FieldDescriptor(
                        messageDescriptor = this@Companion::descriptor,
                        name = "email",
                        number = 3,
                        type = pbandk.FieldDescriptor.Type.Primitive.String(),
                        jsonName = "email",
                        value = tutorial.Person::email
                    )
                )
                add(
                    pbandk.FieldDescriptor(
                        messageDescriptor = this@Companion::descriptor,
                        name = "phones",
                        number = 4,
                        type = pbandk.FieldDescriptor.Type.Repeated<tu