Beautiful logger for http

Proudly created and supported by MadAppGang company.

🚀 v2.0.0 Released!

This is a major release with breaking changes. See the Migration Guide below.

Install:

go get github.com/MadAppGang/httplog/v2@latest

Why?

Every single web framework has a build-in logger already, why do we need on more? The question is simple and the answer is not.

The best logger is structured logger, like Uber Zap. Structure logs are essentials today to help collect and analyze logs with monitoring tools like ElasticSearch or DataDog.

But what about humans? Obviously you can go to your monitoring tool and parse your structured logs. But what about seeing human readable logs in place just in console? Although you can read JSON, it is extremely hard to find most critical information in the large JSON data flow.

Httplog package brings good for humans and log collection systems, you can use tools like zap to create structured logs and see colored beautiful human-friendly output in console in place. Win-win for human and machines 🤖❤️👩🏽‍💻

Nice and clean output is critical for any web framework. Than is why some people use go web frameworks just to get beautiful logs.

This library brings you fantastic http logs to any web framework, even if you use native net/http for that.

But it's better to see once, here the default output you will get with couple of lines of code:

And actual code looks like this:

func main() {
  logger, err := httplog.Logger(happyHandler)
  if err != nil {
    panic(err)
  }
  http.Handle("/happy", logger)

  notFoundLogger, _ := httplog.Logger(http.NotFoundHandler())
  http.Handle("/not_found", notFoundLogger)

  _ = http.ListenAndServe(":3333", nil)
}

All you need is wrap you handler with httplog.Logger and the magic happens.

And this is how the structured logs looks in AWS CloudWatch. As you can see, the color output looks not as good here as in console. But JSON structured logs are awesome 😍

Here is a main features:

• framework agnostic (could be easily integrated with any web framework), you can find examples for:
  • alice
  • chi
  • echo
  • gin
  • goji
  • gorilla mux
  • httprouter
  • mojito
  • negroni
  • native net/http
  • not found yours? let us know and we will add it
• response code using special wrapper
• response length using special wrapper
• can copy response body
• get real user IP for Google App Engine
• get real user IP for CloudFront
• get real user IP for other reverse proxy which implements RFC7239
• customize output format
• has the list of routes to ignore
• build in structure logger integration
• callback function to modify response before write back (add headers or do something)

This framework is highly inspired by Gin logger library, but has not Gin dependencies at all and has some improvements. Httplog has only one dependency at all: github.com/mattn/go-isatty. So it's will not affect your codebase size.

New in v2.0.0

Request Sampling

Log only a percentage of requests to reduce log volume:

logger, _ := httplog.LoggerWithConfig(httplog.LoggerConfig{
    SampleRate: 0.1, // Log 10% of requests
    DeterministicSampling: true, // Same request always sampled the same way
})

Log Level Filtering

Filter logs by HTTP status code:

logger, _ := httplog.LoggerWithConfig(httplog.LoggerConfig{
    MinLevel: httplog.LevelWarn, // Only log 4xx and 5xx responses
})

Async Logging

Reduce request latency with async log writing:

logger, _ := httplog.LoggerWithConfig(httplog.LoggerConfig{
    AsyncLogging: true,
    AsyncBufferSize: 5000,
})
defer logger.Close() // Important! Cleanly shuts down the async goroutine

Request Body Capture

logger, _ := httplog.LoggerWithConfig(httplog.LoggerConfig{
    CaptureRequestBody: true,
})

Color Mode Control

logger, _ := httplog.LoggerWithConfig(httplog.LoggerConfig{
    ColorMode: httplog.ColorForce, // ColorAuto, ColorDisable, ColorForce
})

ConfigBuilder Fluent API

config := httplog.NewConfigBuilder().
    WithName("API").
    WithColorMode(httplog.ColorForce).
    WithSampleRate(0.5).
    WithAsyncLogging(true, 1000).
    Build()

logger, _ := httplog.LoggerWithConfig(config)

Custom format

You can modify formatter as you want. Now there are two formatter available:

• DefaultLogFormatter
• ShortLogFormatter
• HeadersLogFormatter
• DefaultLogFormatterWithHeaders
• BodyLogFormatter
• DefaultLogFormatterWithHeadersAndBody
• RequestHeaderLogFormatter
• DefaultLogFormatterWithRequestHeader
• RequestBodyLogFormatter
• DefaultLogFormatterWithRequestHeadersAndBody
• FullFormatterWithRequestAndResponseHeadersAndBody

And you can combine them using ChainLogFormatter.

Here is an example of formatter in code:

// Short log formatter
shortLoggedHandler, _ := httplog.LoggerWithFormatter(
  httplog.ShortLogFormatter,
)
http.Handle("/short", shortLoggedHandler.Handler(wrappedHandler))

You can define your own log format. Log formatter is a function with a set of precalculated parameters:

// Custom log formatter
customLoggedHandler, _ := httplog.LoggerWithFormatter(
  // formatter is a function, you can define your own
  func(param httplog.LogFormatterParams) string {
    statusColor := param.StatusCodeColor()
    resetColor := param.ResetColor()
    boldRedText := "\033[1;31m"

    return fmt.Sprintf("🥑[I am custom router!!!] %s %3d %s| size: %10d bytes | %s %#v %s 🔮👨🏻‍💻\n",
      statusColor, param.StatusCode, resetColor,
      param.BodySize,
      boldRedText, param.Path, resetColor,
    )
  },
)
http.Handle("/happy_custom", customLoggedHandler.Handler(happyHandler))

For more details and how to capture response body please look in the example app.

params is a type of LogFormatterParams and the following params available for you:

| param | description | | --- | --- | | Request | http.Request instance | | RouterName | when you create logger, you can specify router name | | Timestamp | TimeStamp shows the time after the server returns a response | | StatusCode | StatusCode is HTTP response code | | Latency | Latency is how much time the server cost to process a certain request | | ClientIP | ClientIP calculated real IP of requester, see Proxy for details | | Method | Method is the HTTP method given to the request | | Path | Path is a path the client requests | | BodySize | BodySize is the size of the Response Body | | Context | context.Context from the request | | RequestBody | Request body content (if CaptureRequestBody enabled) | | RequestHeader | Request headers (with masking applied) | | ResponseBody | Response body content (if CaptureResponseBody enabled) | | Level | Log level (Debug, Info, Warn, Error) |

Integrate with structure logger

Good and nice output is good, but as soon as we have so much data about every response it is a good idea to pass it to our application log structured collector.

One of the most popular solution is Uber zap. You can use any structured logger you want, use zap's integration example as a reference.

All you need is create custom log formatter function with your logger integration. This repository has this formatter for zap created and you can use it importing github.com/MadAppGang/httplog/zap:

logger, err := httplog.LoggerWithConfig(httplog.LoggerConfig{
    Formatter: lzap.DefaultZapLogger(zapLogger, zap.InfoLevel, ""),
})
if err != nil {
    panic(err)
}
http.Handle("/happy", logger.Handler(handler))

You can find full-featured example in zap integration folder.

Customize log output destination

You can use any output you need, your output must support io.Writer protocol. After that you need init logger:

buffer := new(bytes.Buffer)
logger, _ := LoggerWithWriter(buffer) //all output is written to buffer
http.Handle("/", logger.Handler(handler))

Care about secrets. Skipping path and masking headers

Some destinations should not be logged. For that purpose logger config has SkipPaths property with array of strings. Each string is a Regexp for path you want to skip. You can write exact path to skip, which would be valid regexp, or you can use regexp power:

logger, _ := LoggerWithConfig(LoggerConfig{
  SkipPaths: []string{
    "/skipped",