<div id="logo" align="center"> <img src="tanuki.png" alt="Tanuki" width="150"/> </div>

Tanuki

Writing web apps are relatively straightforward nowadays especially when there are so many web frameworks to choose from. Just whip out a framework in your favourite programming language and start writing that MVP!

What a lot of people (engineers included) trip over though, is that software is not static and over its lifespan, will grow and evolve. The libraries used in writing them are software too and they will also change and evolve. The programming languages used to write them will change and evolve too. And most importantly, the people writing the software grow and change, and quite often different people end up taking over and continuing the same software.

As you would expect, it takes an experienced and forward thinking engineer (or ‘architect’) to think about these changes, but as you would expect, even the most experienced engineers cannot predict the future. Horror stories of multi-month or even years to just migrate from one version of a framework to another are quite common place. Some even regard dependency hell as inevitable.

There is no silver bullet solution — software is complex and difficult. I’ve been developing software (mostly web apps) for a good 25 years and this problem have been hitting me over the head over and over again. I’ve been toying with different possible answers for a long time and this is my latest attempt.

Sample web app

I've created a sample web app to show how a simple Tanuki web app can be created. Please check out https://github.com/sausheong/pompoko.

How does it work

The basic premise is simple -- remove as much dependency between blocks of execution as possible.

In web apps, blocks of execution are actions that are taken by the system when a user calls a particular URL route. In most web apps, these actions are organised into functions blocks called handlers. A handler takes in a HTTP request, performs the action, and returns a HTTP response. The action might or might not use data from the request but is likely to do so.

A web app is a single piece of software, run in a single process. What if we take each handler and allow it to be developed independently and run as a separate process?

Tanuki is a polyglot web framework that allows developers to write a distributed web app in multiple programming languages. It has an acceptor, which is a HTTP server, that accepts HTTP requests and routes it accordingly to different handlers that can be implemented in different programming languages. The handlers receives a JSON string that has the HTTP request data from the acceptor and returns another JSON string that has the HTTP response data.

There are two types of handlers.

Executable binaries or scripts

Executable binaries or scripts (or bins) are files that are, well, executable. You can normally run them on the command line. As a Tanuki bin handler, it must be able to take process a JSON string (as the first command line argument) and return another JSON string to STDOUT. The input JSON contains HTTP request data and the output JSON has HTTP response data.

TCP socket server

A listener is Tanuki handler that runs as a TCP socket servers. There are two types of listeners:

1. A local listener, which is started by Tanuki and runs in the same host as Tanuki
2. A remote listener, which is not started by Tanuki and Tanuki assumes it is reachable (i.e. it is your responsibility to make sure it’s reachable)

Remote listeners allow Tanuki to be distributed to multiple hosts. This effectively makes the web app distributed!

Installing Tanuki

Tanuki is written in Go. To install it in your platform, you can install Go and get the source:

go get github.com/sausheong/tanuki

There will be downloadable binaries for your platform at a later date.

How to use Tanuki

Once you can have downloaded the source, you can build the command line tool tanuki. This command line tool is used for everything.

go build

With the command line tool, can you create a skeleton structure for your new web application or service:

./tanuki create poko

This will create a new directory named poko with the necessary directories and sample files. To start your new Tanuki web app, go to your new application directory and run this:

./tanuki start

This will start the Tanuki web app at port 8080. You can change the port number or the IP address or hostname as well, just follow the instructions in the tanuki command line tool.

Before you can run any Tanuki web app you need to set up the handlers.

Handler configuration

Handlers are configured in a handlers.yaml file by default. This is how a handler configuration file looks like:

--- # handlers
- method : get
  route  : /_/hello/ruby
  type   : bin
  path   : bin/hello-ruby

- method : post
  route  : /_/hello/go
  type   : bin
  path   : bin/hello-go

- method : get
  route  : /_/hello/ruby/listener
  type   : listener
  local  : true 
  path   : listeners/hello-ruby

- method : get
  route  : /_/hello/go/listener
  type   : listener
  local  : false
  path   : localhost:55771

The configuration is pretty straightforward. The method parameter determines the type of HTTP method to be routed, the route determines the URL route, the type is the type of handler, either a /bin/ or a /listener/ and the path is either a path to the file, or a URL of the remote listener, which includes the hostname and the port. Finally, the local parameter determines if it’s a local or a remote listener.

Handler input and output

HTTP requests

The only input into the handlers (either bins or listeners) is the HTTP request JSON. The below is the Go structs for the JSON (I'm showing you the Go struct because Tanuki is written in Go).

// RequestInfo corresponds to a HTTP 1.1 request
type RequestInfo struct {
	Method           string                 `json:"Method"`
	URL              URLInfo                `json:"URL"`
	Proto            string                 `json:"Proto"`
	Header           map[string][]string    `json:"Header"`
	Body             string                 `json:"Body"`
	ContentLength    int64                  `json:"ContentLength"`
	TransferEncoding []string               `json:"TransferEncoding"`
	Host             string                 `json:"Host"`
	Params           map[string][]string    `json:"Params"`
	Multipart        map[string][]Multipart `json:"Multipart"`
	RemoteAddr       string                 `json:"RemoteAddr"`
	RequestURI       string                 `json:"RequestURI"`
}

// Multipart corresponds to a multi-part file
type Multipart struct {
	Filename    string `json:"Filename"`
	ContentType string `json:"ContentType"`
	Content     string `json:"Content"`
}

// URLInfo corresponds to a URL
type URLInfo struct {
	Scheme   string `json:"Scheme"`
	Opaque   string `json:"Opaque"`
	Host     string `json:"Host"`
	Path     string `json:"Path"`
	RawQuery string `json:"RawQuery"`
	Fragment string `json:"Fragment"`
}

Tanuki provides a Params field, which is a hash (or dictionary) with a string as the key and an array of strings as the value. The Params field is a convenience for Tanuki developers and contains all the parameters sent by the client, including those in the URL or in the case of a POST, also in the forms.

This is an example of the JSON for the HTTP request, a GET request to Tanuki at the URL /_/hello/world.

{
  "Method": "GET",
  "URL": {
    "Scheme": "",
    "Opaque": "",
    "Host": "",
    "Path": "/_/hello/world",
    "RawQuery": "name=sausheong",
    "Fragment": ""
  },
  "Proto": "HTTP/1.1",
  "Header": {
    "Accept": [
      "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
    ],
    "Accept-Encoding": [
      "gzip, deflate"
    ],
    "Accept-Language": [
      "en-sg"
    ],
    "Connection": [
      "keep-alive"
    ],
    "Cookie": [
      "hello=world; mykey=myvalue"
    ],
    "Upgrade-Insecure-Requests": [
      "1"
    ],
    "User-Agent": [
      "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.1.2 Safari/605.1.15"
    ]
  },
  "Body": "",
  "ContentLength": 0,
  "TransferEncoding": null,
  "Host": "localhost:8080",
  "Params": {
    "name": [
      "sausheong"
    ]
  },
  "Multipart": {},
  "RemoteAddr": "[::1]:52340",
  "RequestURI": "/_/hello/world?name=sausheong"
}

You might notice here that if you want to get the name you can either parse the RawQuery or get RequestURI or you can simply get it from Params.

HTTP response

The only response that Tanuki accepts from the handlers is a HTTP response JSON. Here's the Go struct for the JSON.

// ResponseInfo corresponds to a HTTP 1.1 response
type ResponseInfo struct {
	Status int                 `json:"status"`
	Header map[string][]string `json:"header"`
	Body   string              `json:"body"`
}

Here's an example of the response JSON to be sent back to Tanuki.

{
  "status": 200,
  "header": {
    "Content-Length": ["15"],
    "Content-Type": ["text/plain; charset=utf-8"]
  },
  "body": "hello sausheong"
}

Examples

Let's look at some examples. They are pretty simple and almost trivial and does one thing. When the user browses the URL:

https://localhost:8081/_/hello/world?name=sausheong

The handlers should return:

hello sausheong

Bins

Bins are the simplest handler to write and can be written in any programming language than can parse JSON (actually since JSON is simply a string, it, just being able to manipulate strings is good enough) and take in a command line argument and print a JSON string to STDOUT. This is an example of a simple bin