Make your own protoc plugin super-fast with nodejs

I made protoc-plugin to make it easier to write protoc plugins in nodejs.

Protoc is a cool program that google made to generate code from protobuf descriptions.

If you're not familiar with good reasons to use protobuf to describe your protocols, go read this.

On my hackintosh, I got it installed with brew install protobuf. Built-in, it has support for generating a bunch of static implementations of your protobuf interfaces:

--cpp_out=OUT_DIR           Generate C++ header and source.
--csharp_out=OUT_DIR        Generate C# source file.
--java_out=OUT_DIR          Generate Java source file.
--javanano_out=OUT_DIR      Generate Java Nano source file.
--js_out=OUT_DIR            Generate JavaScript source.
--objc_out=OUT_DIR          Generate Objective C header and source.
--php_out=OUT_DIR           Generate PHP source file.
--python_out=OUT_DIR        Generate Python source file.
--ruby_out=OUT_DIR          Generate Ruby source file.

We're going to make a plugin that just outputs info about the user's protobuf files and command-line.

I'm drinking Montucky Cold Snacks and blasting Void Kampf.

start from scratch

Before we start, go setup your npm init defaults for faster drunken code!

Now, make a new project:

mkdir protoc-user-info
npm init # hit enter a bunch
npm i -S protoc-plugin
mkdir bin
touch bin/protoc-gen-user-info
chmod 755 bin/protoc-gen-user-info
atom bin/protoc-gen-user-info

Make bin/protoc-gen-user-info look like this:

#! /usr/bin/env node
const { CodeGeneratorRequest, CodeGeneratorResponse, CodeGeneratorResponseError } = require('protoc-plugin')

CodeGeneratorRequest()
  .then(r => {
    const info = {
      protos: r.toObject().protoFileList,
      request: r
    }
    return [{
      name: 'protoc-user-info.json',
      content: JSON.stringify(info, null, 2)
    }]
  })
  .then(CodeGeneratorResponse())
  .catch(CodeGeneratorResponseError())

demo protobuf

To test it, let's make a simple protobuf file (helloworld.proto, saved in current directory):

syntax = "proto3";
package helloworld;

message Hello {
  string name = 1;
}

run it

Since we haven't installed our plugin on the system yet, we'll need to tell protoc where to find it:

protoc -I . helloworld.proto  --plugin=bin/protoc-gen-user-info --user-info_out=.

You should see protoc-user-info.json, go ahead and have a look at it.

You should see the most relevant stuff in protos.messageTypeList.

further development

This is the more complete way to make a plugin, if you need to see what options your users put into the out flag. If you just want to make a super-simple plugin that takes input protos and outputs some files, see this example. It's a bit simpler.

If you need access more advanced stuff (like --user-info_out=opt1=cool:. to get opt1 as well as path) you can get it in r.array[1] in above example code.

Remember, since the plugin system uses stdout to output files, you will need to use console.error if you want to output stuff (very handy for debugging.) Check out r object above for other tasty bits of info (like original proto files requested, etc)

When you have your plugin working the way you want, add a bit to your package.json that looks like this:

{
  "bin": {
    "protoc-gen-user-info": "bin/protoc-gen-user-info"
  }
}

Now, if your user runs npm install -g, it will get added to their local path, and they won't need to run the --plugin=bin/protoc-gen-user-info part in the above protoc command.

publish it!

Go publish your handy plugin on npm, and blog about it!

echo -e "node_modules\n*.log\n.DS_Store\n*.proto" > .gitignore
git init
git add -A
git commit -am "initial commit"
npm version patch
# make repo somewhere remote, add it as origin
git push --tags
npm publish

further reading

Check back for more protobuf articles. Later, we'll be cooking up some more useful plugins, crafting annotation extensions, and generating hella neat stuff.