Prototool

[![MIT License][mit-img]][mit] [![GitHub Release][release-img]][release] [![Build Status][ci-img]][ci] [![Coverage Status][cov-img]][cov] [![Docker Image][docker-img]][docker] [![Homebrew Package][homebrew-img]][homebrew] [![AUR Package][aur-img]][aur]

Update: We recommend checking out Buf, which is under active development. There are a ton of docs for getting started, including for migration from Prototool.

Protobuf is one of the best interface description languages out there - it's widely adopted, and after over 15 years of use, it's practically bulletproof. However, working with Protobuf and maintaining consistency across your Protobuf files can be a pain - protoc, while being a tool that has stood the test of time, is non-trivial to use, and the Protobuf community has not developed common standards with regards to stub generation. Prototool aims to solve this by making working with Protobuf much simpler.

Prototool lets you:

• Handle installation of protoc and the import of all of the Well-Known Types behind the scenes in a platform-independent manner.
• Standardize building of your Protobuf files with a common configuration.
• Lint your Protobuf files with common linting rules according to Google' Style Guide, Uber's V1 Style Guide, Uber's V2 Style Guide, or your own set of configured lint rules.
• Format your Protobuf files in a consistent manner.
• Create Protobuf files from a template that passes lint, taking care of package naming for you.
• Generate stubs using any plugin based on a simple configuration file, including handling imports of all the Well-Known Types.
• Call gRPC endpoints with ease, taking care of the JSON to binary conversion for you.
• Check for breaking changes on a per-package basis, verifying that your API never breaks.
• Output errors and lint failures in a common file:line:column:message format, making integration with editors possible, Vim integration is provided out of the box.

Prototool accomplishes this by downloading and calling protoc on the fly for you, handing error messages from protoc and your plugins, and using the generated FileDescriptorSets for internal functionality, as well as wrapping a few great external libraries already in the Protobuf ecosystem. Compiling, linting and formatting commands run in around 3/100ths of second for a single Protobuf file, or under a second for a larger number (500+) of Protobuf files.

Table Of Contents

• Installation
• Quick Start
• Full Example
• Configuration
• File Discovery
• Command Overview
  • prototool config init
  • prototool compile
  • prototool generate
  • prototool lint
  • prototool format
  • prototool create
  • prototool files
  • prototool break check
  • prototool descriptor-set
  • prototool grpc
• Tips and Tricks
• Vim Integration
• Stability
• Development
• FAQ
• Special Thanks

Installation

Prototool can be installed on Mac OS X or Linux through a variety of methods.

See install.md for full instructions.

Quick Start

We'll start with a general overview of the commands. There are more commands, and we will get into] usage below, but this shows the basic functionality.

prototool help
prototool lint idl/uber # search for all .proto files recursively, obeying exclude_paths in prototool.yaml or prototool.json files
prototool lint # same as "prototool lint .", by default the current directory is used in directory mode
prototool create foo.proto # create the file foo.proto from a template that passes lint
prototool files idl/uber # list the files that will be used after applying exclude_paths from corresponding prototool.yaml or prototool.json files
prototool lint --list-linters # list all current lint rules being used
prototool lint --list-all-lint-groups # list all available lint groups, currently "google" and "uber"
prototool compile idl/uber # make sure all .proto files in idl/uber compile, but do not generate stubs
prototool generate idl/uber # generate stubs, see the generation directives in the config file example
prototool grpc idl/uber --address 0.0.0.0:8080 --method foo.ExcitedService/Exclamation --data '{"value":"hello"}' # call the foo.ExcitedService method Exclamation with the given data on 0.0.0.0:8080
prototool descriptor-set --include-imports idl/uber # generate a FileDescriptorSet for all files under idl/uber, outputting to stdout, a given file, or a temporary file
prototool break check idl/uber --git-branch master # check for breaking changes as compared to the Protobuf definitions in idl/uber on the master branch

Full Example

See the example directory.

The make command make example runs prototool while installing the necessary plugins.

Configuration

Prototool operates using a config file named either prototool.yaml or prototool.json. Only one of prototool.yaml or prototool.json can exist in a given directory. For non-trivial use, you should have a config file checked in to at least the root of your repository. It is important because the directory of an associated config file is passed to protoc as an include directory with -I, so this is the logical location your Protobuf file imports should start from.

Recommended base config file:

protoc:
  version: 3.11.0
lint:
  group: uber2

See protoc.md for how Prototool handles working with protoc.

The command prototool config init will generate a config file in the current directory with the currently recommended options set.

When specifying a directory or set of files for Prototool to operate on, Prototool will search for config files for each directory starting at the given path, and going up a directory until hitting root. If no config file is found, Prototool will use default values and operate as if there was a config file in the current directory, including the current directory with -I to protoc.

If multiple prototool.yaml or prototool.json files are found that match the input directory or files, an error will be returned.

See etc/config/example/prototool.yaml all available options.

File Discovery

In most Prototool commands, you will see help along the following lines:

$ prototool help lint
Lint proto files and compile with protoc to check for failures.

Usage:
  prototool lint [dirOrFile] [flags]

dirOrFile can take two forms:

• You can specify exactly one directory. If this is done, Prototool goes up until it finds a prototool.yaml or prototool.json file (or uses the current directory if none is found), and then uses this config for all .proto files under the given directory recursively, except for files in the excludes lists in prototool.yaml or prototool.json files.
• You can specify exactly one file. This has the effect as if you specified the directory of this file (using the logic above), but errors are only printed for that file. This is useful for e.g. Vim integration.
• You can specify nothing. This has the effect as if you specified the current directory as the directory.

The idea with "directory builds" is that you often need more than just one file to do a protoc call, for example if you have types in other files in the same package that are not referenced by their fully-qualified name, and/or if you need to know what directories to specify with -I to protoc (by default, the directory of the prototool.yaml or prototool.json file is used).

Command Overview

Let's go over some of the basic commands.

prototool config init

Create a prototool.yaml file in the current directory with the currently recommended options set.

Pass the --document flag to generate a prototool.yaml file with all other options documented and commented out.

Pass the --uncomment flag to generate prototool.yaml file with all options documented but uncommented.

See etc/config/example/prototool.yaml for the config file that prototool config init --uncomment generates.

prototool compile

Compile your Protobuf files, but do not generate stubs. This has the effect of calling protoc with -o /dev/null.

Pass the --dry-run flag to see the protoc commands that Prototool runs behind the scenes.

prototool generate

Compile your Protobuf files and generate stubs according to the rules in your prototool.yaml or prototool.json file.

See etc/config/example/prototool.yaml for all available options. There are special options available for Golang plugins, and plugins that output a single file instead of a set of files. Specifically, you can output a single JAR for the built-in protoc java plugin, and you can output a file with the serialized FileDescriptorSet using the built-in protoc descriptor_set plugin, optionally also calling --include_imports and/or --include_source_info.

Pass the --dry-run flag to see the protoc commands that Prototool runs behind the scenes.

See [example/proto