<!-- Image designed by Jack Lindamood, Licensed under the Creative Commons 3.0 Attributions license, originate from https://github.com/golang-samples/gopher-vector design by Takuya Ueda -->

Circuit

Circuit is an efficient and feature complete Hystrix like Go implementation of the circuit breaker pattern. Learn more about the problems Hystrix and other circuit breakers solve on the Hystrix Wiki. A short summary of advantages are:

• A downstream service failed and all requests hang forever. Without a circuit, your service would also hang forever. Because you have a circuit, you detect this failure quickly and can return errors quickly while waiting for the downstream service to recover.
• Circuits make great monitoring and metrics boundaries, creating common metric names for the common downstream failure types. This package goes further to formalize this in a SLO tracking pattern.
• Circuits create a common place for downstream failure fallback logic.
• Downstream services sometimes fail entirely when overloaded. While in a degraded state, circuits allow you to push downstream services to the edge between absolute failure and mostly working.
• Open/Close state of a circuit is a clear early warning sign of downstream failures.
• Circuits allow you to protect your dependencies from abnormal rushes of traffic.

There are a large number of examples on the godoc that are worth looking at. They tend to be more up to date than the README doc.

Feature set

• No forced goroutines
• recoverable panic()
• Integrated with context.Context
• Comprehensive metric tracking
• Efficient implementation with Benchmarks
• Low/zero memory allocation costs
• Support for Netflix Hystrix dashboards, even with custom circuit transition logic
• Multiple error handling features
• Expose circuit health and configuration on expvar
• SLO tracking
• Customizable state transition logic, allowing complex circuit state changes
• Live configuration changes
• Many tests and examples
• Good inline documentation
• Generatable interface wrapping support with https://github.com/twitchtv/circuitgen
• Support for Additive increase/multiplicative decrease
• Prometheus metrics collector.

Upgrading

See UPGRADE_GUIDE.md for upgrade instructions if you're upgrading from v3 to v4.

Usage

Hello world circuit

This example shows how to create a hello-world circuit from the circuit manager

// Manages all our circuits
h := circuit.Manager{}
// Create a circuit with a unique name
c := h.MustCreateCircuit("hello-world")
// Call the circuit
errResult := c.Execute(context.Background(), func(ctx context.Context) error {
  return nil
}, nil)
fmt.Println("Result of execution:", errResult)
// Output: Result of execution: <nil>

Hello world fallback

This example shows how fallbacks execute to return alternate errors or provide logic when the circuit is open.

// You can create circuits without using the manager
c := circuit.NewCircuitFromConfig("hello-world-fallback", circuit.Config{})
errResult := c.Execute(context.Background(), func(ctx context.Context) error {
	return errors.New("this will fail")
}, func(ctx context.Context, err error) error {
	fmt.Println("Circuit failed with error, but fallback returns nil")
	return nil
})
fmt.Println("Execution result:", errResult)
// Output: Circuit failed with error, but fallback returns nil
// Execution result: <nil>

Running inside a Goroutine

It is recommended to use circuit.Execute and a context aware function. If, however, you want to exit your run function early and leave it hanging (possibly forever), then you can call circuit.Go.

h := circuit.Manager{}
c := h.MustCreateCircuit("untrusting-circuit", circuit.Config{
  Execution: circuit.ExecutionConfig{
    // Time out the context after a few ms
    Timeout: time.Millisecond * 30,
  },
})

errResult := c.Go(context.Background(), func(ctx context.Context) error {
  // Sleep 30 seconds, way longer than our timeout
  time.Sleep(time.Second * 30)
  return nil
}, nil)
fmt.Printf("err=%v", errResult)
// Output: err=context deadline exceeded

Hystrix Configuration

All configuration parameters are documented in config.go. Your circuit open/close logic configuration is documented with the logic. For hystrix, this configuration is in closers/hystrix and well documented on the Hystrix wiki.

This example configures the circuit to use Hystrix open/close logic with the default Hystrix parameters

configuration := hystrix.Factory{
  // Hystrix open logic is to open the circuit after an % of errors
  ConfigureOpener: hystrix.ConfigureOpener{
    // We change the default to wait for 10 requests, not 20, before checking to close
    RequestVolumeThreshold: 10,
    // The default values match what hystrix does by default
  },
  // Hystrix close logic is to sleep then check
  ConfigureCloser: hystrix.ConfigureCloser{
    // The default values match what hystrix does by default
  },
}
h := circuit.Manager{
  // Tell the manager to use this configuration factory whenever it makes a new circuit
  DefaultCircuitProperties: []circuit.CommandPropertiesConstructor{configuration.Configure},
}
// This circuit will inherit the configuration from the example
c := h.MustCreateCircuit("hystrix-circuit")
fmt.Println("This is a hystrix configured circuit", c.Name())
// Output: This is a hystrix configured circuit hystrix-circuit

Enable dashboard metrics

Dashboard metrics can be enabled with the MetricEventStream object. This example creates an event stream handler, starts it, then later closes the handler

// metriceventstream uses rolling stats to report circuit information
sf := rolling.StatFactory{}
h := circuit.Manager{
  DefaultCircuitProperties: []circuit.CommandPropertiesConstructor{sf.CreateConfig},
}
es := metriceventstream.MetricEventStream{
  Manager: &h,
}
go func() {
  if err := es.Start(); err != nil {
    log.Fatal(err)
  }
}()
// ES is a http.Handler, so you can pass it directly to your mux
http.Handle("/hystrix.stream", &es)
// ...
if err := es.Close(); err != nil {
  log.Fatal(err)
}
// Output:

Enable expvar

If you wanted to publish hystrix information on Expvar, you can register your manager.

h := circuit.Manager{}
expvar.Publish("hystrix", h.Var())

Custom metrics

Implement interfaces CmdMetricCollector or FallbackMetricCollector to know what happens with commands or fallbacks. Then pass those implementations to configure.

config := circuit.Config{
  Metrics: circuit.MetricsCollectors{
    Run: []circuit.RunMetrics{
      // Here is where I would insert my custom metric collector
    },
  },
}
circuit.NewCircuitFromConfig("custom-metrics", config)

Panics

Code executed with Execute does not spawn a goroutine and panics naturally go up the call stack to the caller. This is also true for Go, where we attempt to recover and throw panics on the same stack that calls Go. This example will panic, and the panic can be caught up the stack.

h := circuit.Manager{}
c := h.MustCreateCircuit("panic_up")

defer func() {
 r := recover()
 if r != nil {
   fmt.Println("I recovered from a panic", r)
 }
}()
c.Execute(context.Background(), func(ctx context.Context) error {
 panic("oh no")
}, nil)
// Output: I recovered from a panic oh no

Runtime configuration changes

Most configuration properties on the Hystrix Configuration page that say they are modifyable at runtime can be changed on the Circuit in a thread safe way. Most of the ones that cannot are related to stat collection.

This example shows how to update hystrix configuration at runtime.

// Start off using the defaults
configuration := hystrix.ConfigFactory{}
h := circuit.Manager{
  // Tell the manager to use this configuration factory whenever it makes a new circuit
  DefaultCircuitProperties: []circuit.CommandPropertiesConstructor{configuration.Configure},
}
c := h.MustCreateCircuit("hystrix-circuit")
fmt.Println("The default sleep window", c.OpenToClose.(*hystrix.Closer).Config().SleepWindow)
// This configuration update function is thread safe.  We can modify this at runtime while the circuit is active
c.OpenToClose.(*hystrix.Closer).SetConfigThreadSafe(hystrix.ConfigureCloser{
  SleepWindow: time.Second * 3,
})
fmt.Println("The new sleep window", c.OpenToClose.(*hystrix.Closer).Config().SleepWindow)
// O