❤️ PowerProto is actively maintained! Any questions in use can be directly raised issue, I will respond to you as fast as possible. If you think the project is helpful to you, please give me a star to encourage me!

PowerProto

English | 中文

PowerProto is used to solve the following three main problems:

1. lower the usage threshold and usage cost of gRPC.
2. solve the version control problem of protoc and its related plugins (such as protoc-gen-go, protoc-gen-grpc-gateway).
3. efficiently manage the compilation of proto to achieve multi-platform compatibility, one-click installation and compilation.

• PowerProto
  • 🎉 Features
  • Installation and Dependencies
    • I. Installation via Go
    • II. out-of-the-box version
  • Command Introduction
    • I. Initial Config
    • II. Tidy Config
    • III. Compiling Proto files
    • IV. View environment variables
  • Examples
  • Config File
    • Definition
      • Matching patterns and working directory
      • Multi-config
    • PostAction
      • 1. copy
      • 2. move
      • 3. remove
      • 4. replace

🎉 Features

1. one-click installation and multi-version management of protoc.
2. one-click installation and multi-version management of protoc related plugins (such as protoc-gen-go).
3. manage the compilation of proto through config file instead of shell script to improve readability and compatibility.
4. bootstrap generation of config files, cross-platform compatibility, a config can be compiled in multiple platforms with one click.
5. support batch and recursive compilation of proto files to improve efficiency.
6. cross-platform support PostAction, you can perform some routine operations (such as replacing "omitempty" in all generated files) after the compilation.
7. support PostShell, execute specific shell scripts after the compilation.
8. one-click installation and version control of google apis and gogo protobuf etc。

Installation and Dependencies

1. The current version of PowerProto relies on go(>=1.16) and git (in the future it may use CDN to pull built binaries directly), please make sure the runtime environment contains these two commands.
2. protoc download source is Github, PowerProto respects HTTP_PROXY, HTTPS_PROXY environment variables when downloading protoc, if you encounter network problems, please configure your own proxy.
3. When querying the version list of protoc, git ls-remote is used for github.com, if you encounter network problems, please configure the proxy for git by yourself.
4. In the current version, downloading and querying plugin versions rely on the go command, so if you encounter network problems, please configure the GOPROXY environment variable yourself.
5. By default, user directory/.powerproto is used as the installation directory, which is used to place the downloaded plug-ins and global config.
6. If you think the name powerproto is too long, you can alias it into a simpler name to improve the input efficiency, for example, no one will mind if you call it pp.

I. Installation via Go

Installation can be performed by executing the following command directly:

go install github.com/storyicon/powerproto/cmd/powerproto@latest

II. out-of-the-box version

You can download the out-of-the-box version via the Github Release Page

Command Introduction

You can view help with powerproto -h, e.g.

powerproto -h
powerproto init -h
powerproto tidy -h
powerproto build -h
powerproto env -h

It has the advantage that the documentation on the command line is always consistent with your binary version.

I. Initial Config

The config can be initialized with the following command.

powerproto init

II. Tidy Config

The config can be tidied with the following command.

powerproto tidy

It will search for a config file named powerproto.yaml from the current directory to the parent directory, and will read and tidy the config.

You can also specify which config file to tidy.

powerproto tidy [the path of proto file]

Tidy the config consists of two main operations:

1. replacing the latest in the version with the real latest version number by querying.
2. install all dependencies defined in the config file.

Supports entering debug mode by appending the -d argument to see more detailed logs.

III. Compiling Proto files

The Proto file can be compiled with the following command.

// Compile the specified proto file
powerproto build xxxx.proto

// Compile all the proto files in the current directory
powerproto build .

// Compile all proto files in the current directory recursively, including subfolders.
powerproto build -r .

The execution logic is that for each proto file, the powerproto.yaml config file will be searched from the directory where the proto file is located to the ancestor directory:

1. For the found config file, match it with the scope in it and use it if it matches.
2. Check and install the dependencies declared in the config file.
3. Compile the proto file according to the plugins, protoc, options, importPaths and other configs in the config file。 After all the proto files are compiled, if you specify the -p argument, PostAction and PostShell will also be executed.

Note: The default working directory of PowerProto is the directory where the proto file matches to the config file, it is equivalent to the directory where you execute the protoc command. You can change it via protocWorkDir in the config file.

Supports entering debug mode by appending the -d argument to see more detailed logs.

Supports entering dryRun mode by appending the -y argument, in this mode the commands are not actually executed, but just printed out, which is very useful for debugging.

IV. View environment variables

If your command keeps getting stuck in a certain state, there is a high probability that there is a network problem.

You can check if the environment variables are configured successfully with the following command:

powerproto env

Examples

For example, you have the following file structure in the /mnt/data/hello directory:

$ pwd
/mnt/data/hello

$ tree
./apis
└── hello.proto
powerproto.yaml

The contents of the powerproto.yaml file (you can easily generate the config file with the powerproto init command) are:

scopes:
    - ./
protoc: latest
protocWorkDir: ""
plugins:
    protoc-gen-go: google.golang.org/protobuf/cmd/protoc-gen-go@latest
    protoc-gen-go-grpc: google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
repositories:
    GOOGLE_APIS: https://github.com/googleapis/googleapis@75e9812478607db997376ccea247dd6928f70f45
options:
    - --go_out=.
    - --go_opt=paths=source_relative
    - --go-grpc_out=.
    - --go-grpc_opt=paths=source_relative
importPaths:
    - .
    - $GOPATH
    - $POWERPROTO_INCLUDE
    - $GOOGLE_APIS/github.com/googleapis/googleapis
postActions: []
postShell: ""

Execute in any directory:

powerproto build -r /mnt/data/hello/apis

You can get the compiled file:

$ pwd
/mnt/data/hello

$ tree
./apis
├── hello.pb.go
├── hello.proto
└── hello_grpc.pb.go
powerproto.yaml

It is equivalent to if you were in the directory where powerproto.yaml is located and executed:

$POWERPROTO_HOME/protoc/3.17.3/protoc --go_out=. \
--go_opt=paths=source_relative \
--go-grpc_out=. \
--go-grpc_opt=paths=source_relative \
--proto_path=/mnt/data/hello \
--proto_path=$GOPATH \
--proto_path=$POWERPROTO_HOME/include \
--proto_path=$POWERPROTO_HOME/gits/75e9812478607db997376ccea247dd6928f70f45/github.com/googleapis/googleapis \
--plugin=protoc-gen-go=$POWERPROTO_HOME/plugins/google.golang.org/protobuf/cmd/protoc-gen-go@v1.27.1/protoc-gen-go \
--plugin=protoc-gen-go-grpc=$POWERPROTO_HOME/plugins/google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1.0/protoc-gen-go-grpc \
/mnt/data/hello/apis/hello.proto

More examples can be found in examples.

Config File

The config file is used to describe the versions of various dependencies and parameters when compiling the proto file.

It can be easily initialized with powerproto init.

Definition

Take the following config file as an example:

# required. scopes is used to define scopes. 
# i.e. which directories in the project the current config item is valid for
scopes:
    - ./
# required. the version of protoc.
# you can fill in the 'latest', will be automatically converted to the latest version
protoc: 3.17.3
# optional. The working directory for executing the protoc command, 
# the default is the directory where the config file is located.
# support mixed environment variables in path, such as $GOPATH
protocWorkDir: ""
# optional. define dependent Git repositories
# Generally used for dependency control of public protobuf libraries
repositories:
    # Definition depends on the 27156597fdf