Tessera

Tessera is a Go library for building tile-based transparency logs (tlogs). It is the logical successor to the approach [Trillian v1][] takes in building and operating logs.

The implementation and its APIs bake-in current best-practices based on the lessons learned over the past decade of building and operating transparency logs in production environments and at scale.

Tessera was introduced at the Transparency.Dev summit in October 2024. Watch Introducing Tessera for all the details, but here's a summary of the high level goals:

• [tlog-tiles API][] and storage
• Support for both cloud and on-premises infrastructure
  • GCP
  • AWS
  • POSIX
• Make it easy to build and deploy new transparency logs on supported infrastructure
  • Library instead of microservice architecture
  • No additional services to manage
  • Lower TCO for operators compared with Trillian v1
• Fast sequencing and integration of entries
• Optional functionality which can be enabled for those ecosystems/logs which need it (only pay the cost for what you need):
  • "Best-effort" de-duplication of entries
  • Synchronous integration
• Broadly similar write-throughput and write-availability, and potentially far higher read-throughput and read-availability compared to Trillian v1 (dependent on underlying infrastructure)
• Enable building of arbitrary log personalities, including support for the peculiarities of a [Static CT API][] compliant log.

The main non-goal is to support transparency logs using anything other than the [tlog-tiles API][]. While it is possible to deploy a custom personality in front of Tessera that adapts the tlog-tiles API into any other API, this strategy will lose a lot of the read scaling that Tessera is designed for.

Table of Contents

• Status
• Roadmap
• Concepts
• Usage
  • Getting Started
  • Writing Personalities
• Features
• Lifecycles
• Contributing
• License
• Contact

Status

Tessera is under active development, and is considered production ready since the Beta release. See the table below for details.

Storage drivers

| Driver | Appender | Migration | Antispam | Garbage Collection | | ----------------------- | :------: | :-------: | :------: | :----------------: | | Amazon Web Services | ✅ | ⚠️ | ✅ | ✅ | | Google Cloud Platform | ✅ | ⚠️ | ✅ | ✅ | | POSIX filesystem | ✅ | ⚠️ | ✅ | ✅ |

[!Note] Please get in touch if you are interested in using any of the features or drivers held back in BETA above.

Users of GCP, AWS, and POSIX are welcome to try the relevant Getting Started guide.

Roadmap

Tessera is generally available and production ready. The following items are planned for future releases:

| # | Step | Status | | :-: | --------------------------------------------------------- | :----: | | 1 | Drivers for GCP, AWS, and POSIX | ✅ | | 2 | [tlog-tiles API][] support | ✅ | | 3 | Example code and terraform scripts for easy onboarding | ✅ | | 4 | Stable API | ✅ | | 5 | Data migration between releases | ✅ | | 6 | Data migration between drivers | ✅ | | 7 | Witness support | ✅ | | 8 | Monitoring and metrics | ✅ | | 9 | Production ready | ✅ | | 10 | Mirrored logs ([#576][]) | ⚠️ | | 11 | Preordered logs ([#575][]) | ❌ | | 12 | Trillian v1 to Tessera migration ([#577][]) | ❌ | | N | Fancy features (to be expanded upon later) | ❌ |

What’s happening to Trillian v1?

[Trillian v1][] is still in use in production environments by multiple organisations in multiple ecosystems, and is likely to remain so for the mid-term.

New ecosystems, or existing ecosystems looking to evolve, should strongly consider planning a migration to Tessera and adopting the patterns it encourages.

[!Tip] To achieve the full benefits of Tessera, logs must use the [tlog-tiles API][].

Concepts

This section introduces concepts and terms that will be used throughout the user guide.

Sequencing

When data is added to a log, it is first stored in memory for some period (this can be controlled via the batching options). If the process dies in this state, the entry will be lost.

Once a batch of entries is processed by the sequencer, the new data will transition from a volatile state to one where it is durably assigned an index. If the process dies in this state, the entry will be safe, though it will not be available through the read API of the log until the leaf has been Integrated. Once an index number has been issued to a leaf, no other data will ever be issued the same index number. All index numbers are contiguous and start from 0.

[!IMPORTANT] Within a batch, there is no guarantee about which order index numbers will be assigned. The only way to ensure that sequential calls to Add are given sequential indices is by blocking until a sequencing batch is completed. This can be achieved by configuring a batch size of 1, though this will make sequencing expensive!

Integration

Integration is a background process that happens when a Tessera lifecycle object has been created. This process takes sequenced entries and merges them into the log. Once this process has been completed, a new entry will:

• Be available via the read API at the index that was returned from sequencing
• Have Merkle tree hashes that commit to this data being included in the tree

Publishing

Publishing is a background process that creates a new Checkpoint for the latest tree. This background process runs periodically (configurable via WithCheckpointInterval and WithCheckpointRepublishInterval) and performs the following steps:

1. Create a new Checkpoint and sign it with the signer provided by WithCheckpointSigner
2. Contact witnesses and collect enough countersignatures to satisfy any witness policy configured by WithWitnesses
3. If the witness policy is satisfied, make this new Checkpoint public available

An entry is considered published once it is committed to by a published Checkpoint (i.e. a published Checkpoint's size is larger than the entry's assigned index). Due to the nature of append-only logs, all Checkpoints issued after this point will also commit to inclusion of this entry.

Usage

Getting Started

The best place to start is the codelab. This will walk you through setting up your first log, writing some entries to it via HTTP, and inspecting the contents.

Take a look at the example personalities in the /cmd/ directory:

• posix: example of operating a log backed by a local filesystem
  • This example runs an HTTP web server that takes arbitrary data and adds it to a file-based log.
• gcp: example of operating a log running in GCP.
  • This example can be deployed via terraform, see the deployment instructions.
• aws: example of operating a log running on AWS.
  • This example can be deployed via terraform, see the deployment instructions.
• posix-oneshot: example of a command line tool to add entries to a log stored on the local filesystem
  • This example is not a long-lived process; running the command integrates entries into the log which lives only as files.

The main.go files for each of these example personalities try to strike a balance when demonstrating features of Tessera between simplicity, and demonstrating best practices. Please raise issues against the repo, or chat to us in Slack if you have ideas for making the examples more accessible!

Writing Personalities

Introduction

Tessera is a library written in Go. It is designed to efficiently serve logs that allow read access via the [tlog-tiles API][]. The code you write that calls Tessera is referred to as a personali