QuickBuffers - Fast Protocol Buffers without Allocations

QuickBuffers is a Java implementation of Google's Protocol Buffers that has been developed for low latency use cases in zero-allocation environments. It has no external dependencies, and the API follows Protobuf-Java where feasible to simplify migration.

The main highlights are

Allocation-free in steady state. All parts of the API are mutable and reusable.
No reflections. GraalVM native-images and R8/ProGuard obfuscation (config) are supported out of the box
Faster encoding and decoding speed
Smaller code size than protobuf-javalite
Built-in JSON marshalling compliant with the proto3 mapping
Improved order for optimized sequential memory access
Optional accessors as an opt-in feature (java8)

QuickBuffers passes all proto2 conformance tests and is compatible with all Java versions from 6 through 20 as well as Android. Proto3 messages can be generated and are wire compatible, but so far the behavioral differences have not been explicitly added due to some proto3 design decisions that have kept us from using it. Current limitations include

Services are not implemented
Extensions are embedded directly into the extended message, so support is limited to generation time.
The proto files for well-known proto3 types such as timestamp.proto or duration.protoneed to be included manually. Their special cased JSON representations are not implemented.
Unsigned integer types are JSON encoded as signed integer numbers

Getting started

In order to use QuickBuffers you need to generate messages and add the corresponding runtime dependency. The runtime can be found at the Maven coordinates below.

<properties>
  <quickbuf.version>1.4</quickbuf.version>
  <quickbuf.options>indent=4,allocation=lazy,extensions=embedded</quickbuf.options>
</properties>

<dependency>
  <groupId>us.hebi.quickbuf</groupId>
  <artifactId>quickbuf-runtime</artifactId>
  <version>${quickbuf.version}</version>
</dependency>

The message generator protoc-gen-quickbuf is set up as a plugin for the protocol buffers compiler protoc. You can install one of the pre-built packages and run:

protoc-quickbuf --quickbuf_out=${options>:<outputDir> <protoFiles>

or use a protoc-gen-quickbuf-${version}-${arch}.exe plugin binary with an absolute pluginPath:

protoc --plugin-protoc-gen-quickbuf=${exePath} --quickbuf_out=${options>:<outputDir> <protoFiles>

or build messages in Maven using the protoc-jar-maven-plugin:

<!-- Downloads protoc w/ plugin and generates messages -->
<!-- Default settings expect .proto files to be in src/main/protobuf -->
<plugin>  
  <groupId>com.github.os72</groupId>
  <artifactId>protoc-jar-maven-plugin</artifactId>
  <version>3.11.4</version>
  <executions>
    <execution>
      <phase>generate-sources</phase>
      <goals>
        <goal>run</goal>
      </goals>
      <configuration>
        <protocVersion>3.21.12</protocVersion>

        <outputTargets>
          <outputTarget>
            <type>quickbuf</type>
            <pluginArtifact>us.hebi.quickbuf:protoc-gen-quickbuf:${quickbuf.version}</pluginArtifact>
            <outputOptions>${quickbuf.options}</outputOptions>
          </outputTarget>
        </outputTargets>

      </configuration>
    </execution>
  </executions>
</plugin>

The generator features several options that can be supplied as a comma-separated list. The default values are marked bold.

| Option | Value | Description | |:-------------------------|:---------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | indent | 2, 4, 8, tab | sets the indentation in generated files | | replace_package | (pattern)=replacement | replaces the Java package of the generated messages to avoid name collisions with messages generated by --java_out. | | input_order | quickbuf, number, none | improves decoding performance when parsing messages that were serialized in a known order. number matches protobuf-java, and none disables this optimization (not recommended). | | output_order | quickbuf, number | number matches protobuf-java serialization to pass conformance tests that require binary equivalence (not recommended). | | store_unknown_fields | false, true | generates code to retain unknown fields that were encountered during parsing. This allows messages to be routed without losing information, even if the schema is not fully known. Unknown fields are stored in binary form and are ignored in equality checks. | | enforce_has_checks | false, true | throws an exception when accessing fields that were not set |
| allocation | eager, lazy, lazymsg | changes the allocation strategy for nested types. eager allocates up-front and results in fewer runtime-allocations, but it may be wasteful and prohibits recursive type declarations. lazy waits until the field is actually needed. lazymsg acts lazy for nested messages, and eager for everything else. | | extensions | disabled, embedded | embedded adds extensions from within a single protoc call directly to the extended message. This requires extensions to be known at generation time. Some plugins may do a separate request per file, so it may require an import to combine multiple files. | | java8_optional | false, true | creates tryGet methods that are short for return if(hasField()) ? Optional.of(getField()) : Optional.absent(). Requires a runtime with Java 8 or higher. |
| gen_descriptors | false, true | creates descriptor information for integrating with reflection in existing tools

Reading and writing messages

We tried to keep the public API as close to Google's protobuf-java as possible, so most use cases should require very few changes. The Java related file options are all supported and behave the same way.

// .proto definition
message RootMessage {
  optional string text = 1;

QuickBuffers

Install / Use

README

QuickBuffers - Fast Protocol Buffers without Allocations

Getting started

Reading and writing messages