webhookd

A very simple webhook server to launch shell scripts.

At a glance

Installation

Run the following command:

$ go install github.com/ncarlier/webhookd@latest

Or download the binary for your architecture:

$ sudo curl -s https://raw.githubusercontent.com/ncarlier/webhookd/master/install.sh | bash
or
$ curl -sf https://gobinaries.com/ncarlier/webhookd | sh

Or use Docker:

$ docker run -d --name=webhookd \
  -v ${PWD}/scripts:/scripts \
  -p 8080:8080 \
  ncarlier/webhookd

Note: The official Docker image is lightweight and allows to run simple scripts but for more advanced needs you can use the ncarlier/webhookd:edge-distrib image. For example, with this distrib image, you can interact with your Docker daemon using Docker CLI or Docker Compose.

Or use APT:

Finally, it is possible to install Webhookd using the Debian packaging system through this custom repository.

Note: Custom configuration variables can be set into /etc/webhookd.env file. Systemd service is already set and enabled, you just have to start it with systemctl start webhookd.

Configuration

Webhookd can be configured by using command line parameters or by setting environment variables.

Type webhookd -h to display all parameters and related environment variables.

All configuration variables are described in etc/default/webhookd.env file.

Usage

Directory structure

Webhooks are simple scripts within a directory structure.

By default inside the ./scripts directory. You can change the default directory using the WHD_HOOK_SCRIPTS environment variable or -hook-scripts parameter.

Example:

/scripts
|--> /github
  |--> /build.sh
  |--> /deploy.sh
|--> /push.js
|--> /echo.sh
|--> ...

Note: Webhookd is able to run any type of file in this directory as long as the file is executable. For example, you can execute a Node.js file if you give execution rights to the file and add the appropriate #! header (in this case: #!/usr/bin/env node).

You can find sample scripts in the example folder. In particular, examples of integration with Gitlab and Github.

Webhook call

The directory structure defines the webhook URL.

You can omit the script extension. If you do, webhookd will search by default for a .sh file. You can change the default extension using the WHD_HOOK_DEFAULT_EXT environment variable or -hook-default-ext parameter. If the script exists, the output will be sent to the HTTP response.

Depending on the HTTP request, the HTTP response will be a HTTP 200 code with the script's output in real time (streaming), or the HTTP response will wait until the end of the script's execution and return the output (truncated) of the script as well as an HTTP code relative to the script's output code.

The streaming protocol depends on the HTTP request:

• Server-sent events is used when Accept HTTP header is equal to text/event-stream.
• Chunked Transfer Coding is used when X-Hook-Mode HTTP header is equal to chunked. It's the default mode. You can change the default mode using the WHD_HOOK_DEFAULT_MODE environment variable or -hook-default-mode parameter.

If no streaming protocol is needed, you must set X-Hook-Mode HTTP header to buffered. The HTTP response will block until the script is over:

• Sends script output limited to the last 100 lines. You can modify this limit via the HTTP header X-Hook-MaxBufferedLines.
• Convert the script exit code to HTTP code as follow:
  • 0: 200 OK
  • Between 1 and 99: 500 Internal Server Error
  • Between 100 and 255: Add 300 to get HTTP code between 400 and 555

Remember: a process exit code is between 0 and 255. 0 means that the execution is successful.

Example:

The script: ./scripts/foo/bar.sh

#!/bin/bash

echo "foo foo foo"
echo "bar bar bar"

exit 118

Streamed output using Server-sent events:

$ curl -v --header "Accept: text/event-stream" -XGET http://localhost:8080/foo/bar
< HTTP/1.1 200 OK
< Content-Type: text/event-stream
< Transfer-Encoding: chunked
< X-Hook-Id: 8

data: foo foo foo

data: bar bar bar

error: exit status 118

Streamed output using Chunked Transfer Coding:

$ curl -v -XPOST --header "X-Hook-Mode: chunked" http://localhost:8080/foo/bar
< HTTP/1.1 200 OK
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
< X-Hook-Id: 7

foo foo foo
bar bar bar
error: exit status 118

Blocking HTTP request:

$ curl -v -XPOST --header "X-Hook-Mode: buffered" http://localhost:8080/foo/bar
< HTTP/1.1 418 I m a teapot
< Content-Type: text/plain; charset=utf-8
< X-Hook-Id: 9

foo foo foo
bar bar bar
error: exit status 118

Note that in this last example the HTTP response is equal to exit code + 300 : 418 I'm a teapot.

Webhook parameters

You have several ways to provide parameters to your webhook script:

• URL request parameters are converted to script variables
• HTTP headers are converted to script variables
• Request body (depending the Media Type):
  • application/x-www-form-urlencoded: keys and values are converted to script variables
  • text/* or application/json: payload is transmitted to the script as first parameter.

Note: Variable name follows "snakecase" naming convention. Therefore the name can be altered. ex: CONTENT-TYPE will become content_type.

Webhookd adds some additional parameters to the script:

• hook_id: hook ID (auto-increment)
• hook_name: hook name
• hook_method: HTTP request method
• x_forwarded_for: client IP
• x_webauth_user: username if authentication is enabled

Example:

The script:

#!/bin/bash

echo "Hook information: name=$hook_name, id=$hook_id, method=$hook_method"
echo "Query parameter: foo=$foo"
echo "Header parameter: user-agent=$user_agent"
echo "Script parameters: $1"

The result:

$ curl --data @test.json -H 'Content-Type: application/json' http://localhost:8080/echo?foo=bar
Hook information: name=echo, id=1, method=POST
Query parameter: foo=bar
Header parameter: user-agent=curl/7.52.1
Script parameter: {"message": "this is a test"}

Webhook timeout configuration

By default a webhook has a timeout of 10 seconds. This timeout is globally configurable by setting the environment variable: WHD_HOOK_TIMEOUT (in seconds).

You can override this global behavior per request by setting the HTTP header: X-Hook-Timeout (in seconds).

Example:

$ curl -H "X-Hook-Timeout: 5" http://localhost:8080/echo?foo=bar

Webhook logs

As mentioned above, web hook logs are stream in real time during the call. However, you can retrieve the logs of a previous call by using the hook ID: http://localhost:8080/<NAME>/<ID>

The hook ID is returned as an HTTP header with the Webhook response: X-Hook-ID

Example:

$ # Call webhook
$ curl -v http://localhost:8080/echo?foo=bar
...
< HTTP/1.1 200 OK
< Content-Type: text/plain
< X-Hook-Id: 2
...
$ # Retrieve logs afterwards
$ curl http://localhost:8080/echo/2

If needed, you can also redirect hook logs to the server output (configured by the WHD_LOG_MODULES=hook environment variable).

Post hook notifications

The output of the script is collected and stored into a log file (configured by the WHD_HOOK_LOG_DIR environment variable).

Once the script is executed, you can send the result and this log file to a notification channel. Currently, only two channels are supported: Email and HTTP.

Notifications configuration can be done as follow:

$ export WHD_NOTIFICATION_URI=http://requestb.in/v9b229v9
$ # or
$ webhookd --notification-uri=http://requestb.in/v9b229v9

Note: Only the output of the script prefixed by "notify:" is sent to the notification channel. If the output does not contain a prefixed line, no notification will be sent.

Example:

#!/bin/bash

echo "notify: Hello World" # Will be notified
echo "Goodbye"             # Will not be notified

You can override the notification prefix by adding prefix as a query parameter to the configuration URL.

Example: http://requestb.in/v9b229v9?prefix="foo:"

HTTP notification

Configuration URI: http://example.org

Options (using query parameters):

• prefix: Prefix to filter output log

The following JSON payload is POST to the target URL:

{
  "id": "42",
  "name": "echo",
  "text": "foo\nbar...\n",
  "error": "Error cause... if present",
}

Note: that because the payload have a text attribute, you can use a Mattermost, Slack or Discord webhook endpoint.

Email notification

Configuration URI: mailto:foo@bar.com

Options (using query parameters):

• prefix: Prefix to filter output log
• smtp: SMTP host to use (by default: localhost:25)
• username: SMTP username (not set by default)
• password: SMTP password (not set by default)
• conn: SM