Diplomat

A HTTP Ruby API for Consul

FAQ

What's Diplomat for?

Diplomat allows any ruby application to interact with Consul's distributed key value store, and also receive information about services currently available in the Consul cluster.

Does it work in rails?

Yup! In fact, we're using it in all of our rails production apps instead of any previous case where it'd be right to use environment variables according to 12Factor configuration principals. This gives us the ability to scale up without making any changes to the actual project codebase, and to move applications around the cluster with ease.

Here's what a production database.yml file might look like:

<% if Rails.env.production? %>
production:
  adapter:            postgresql
  encoding:           unicode
  host:               <%= Diplomat::Service.get('postgres').Address %>
  database:           <%= Diplomat::Kv.get('project/db/name') %>
  pool:               5
  username:           <%= Diplomat::Kv.get('project/db/user') %>
  password:           <%= Diplomat::Kv.get('project/db/pass') %>
  port:               <%= Diplomat::Service.get('postgres').ServicePort %>
<% end %>

Why would I use Consul over ZooKeeper, Doozerd, etcd, Nagios, Sensu, SmartStack, SkyDNS, Chef, Puppet, Ansible, etc?

Read up what makes Consul different here

How do I install Consul?

See here. I managed to roll it out on my production machines with the help of Ansible in one working day.

Which versions of Ruby does Diplomat support? Where did my ruby 1.9 compatibility go?

Check out GitHub Actions to see which versions of ruby we currently test when we're making builds.

We've dropped ruby 1.9 support. You can still depend on Diplomat by directly using the ruby-1.9-compatible branch on github, although be advised it's not actively maintained anymore.

ERB templating

It is possible to inject diplomat data into .erb files (such as in chef), but you could also have a look at consul-templaterb that is highly optimized for ERB templating with very hi parallelism and good optimized performance for large clusters.

Usage

The most up to date place to read about the API is here.

Here's a few examples of how diplomat works:

Key Values

Setting

Setting the value of a key is easy as pie:

foo = Diplomat::Kv.put('foo', 'bar')
# => "bar"

Getting

Getting the value of a key is just as simple:

foo = Diplomat::Kv.get('foo')
# => "bar"

Or retrieve a value from another datacenter:

foo = Diplomat::Kv.get('foo', :dc => 'dc-west')
# => "baz"

You can also retrieve values recursively:

Diplomat::Kv.put('foo/a', 'lorem')
Diplomat::Kv.put('foo/b', 'ipsum')
Diplomat::Kv.put('foo/c', 'dolor')

Diplomat::Kv.get('foo/', recurse: true)
# => [{:key=>"foo/a", :value=>"lorem"}, {:key=>"foo/b", :value=>"ipsum"}, {:key=>"foo/c", :value=>"dolor"}]

You can also use get_all to retrieve values recursively with a consistent return type:

Diplomat::Kv.put('foo/a', 'lorem')
Diplomat::Kv.put('foo/b', 'ipsum')
Diplomat::Kv.put('foo/c', 'dolor')

Diplomat::Kv.get('foo/', recurse: true)
# => [{:key=>"foo/a", :value=>"lorem"}, {:key=>"foo/b", :value=>"ipsum"}, {:key=>"foo/c", :value=>"dolor"}]
Diplomat::Kv.get_all('foo/')
# => [{:key=>"foo/a", :value=>"lorem"}, {:key=>"foo/b", :value=>"ipsum"}, {:key=>"foo/c", :value=>"dolor"}]

Diplomat::Kv.put('bar/a', 'lorem')

Diplomat::Kv.get('bar/', recurse: true)
# => "lorem"
Diplomat::Kv.get_all('bar/')
# => [{:key=>"bar/a", :value=>"lorem"}]

Or list all available keys:

Diplomat::Kv.get('/', :keys => true) # => ['foo/a', 'foo/b']

You can convert the consul data to a ruby hash

Diplomat::Kv.put('foo/a', 'lorem')
Diplomat::Kv.put('foo/b', 'ipsum')
Diplomat::Kv.put('foo/c', 'dolor')

Diplomat::Kv.get('foo/', recurse: true, convert_to_hash: true)
# => {"foo"=>{"a"=>"lorem", "b"=>"ipsum", "c"=>"dolor"}}

Nodes

Getting

Look up a node:

foo_service = Diplomat::Node.get('foo')
# => {"Node"=>{"Node"=>"foobar", "Address"=>"10.1.10.12"}, "Services"=>{"consul"=>{"ID"=>"consul", "Service"=>"consul", "Tags"=>nil, "Port"=>8300}, "redis"=>{"ID"=>"redis", "Service"=>"redis", "Tags"=>["v1"], "Port"=>8000}}}

Get all nodes:

nodes = Diplomat::Node.get_all
# => [#<OpenStruct Address="10.1.10.12", Node="foo">, #<OpenStruct Address="10.1.10.13", Node="bar">]

Get all nodes for a particular datacenter

nodes = Diplomat::Node.get_all({ :dc => 'My_Datacenter' })
# => [#<OpenStruct Address="10.1.10.12", Node="foo">, #<OpenStruct Address="10.1.10.13", Node="bar">]

Register a node:

Diplomat::Node.register({ :Node => "app1", :Address => "10.0.0.2" })
# => true

De-register a node:

Diplomat::Node.deregister({ :Node => "app1", :Address => "10.0.0.2" })
# => true

Services

Getting

Looking up a service is easy as pie:

foo_service = Diplomat::Service.get('foo')
# => #<OpenStruct Node="hotel", Address="1.2.3.4", ServiceID="hotel_foo", ServiceName="foo", ServiceTags=["foo"], ServicePort=5432>

Or if you have multiple nodes per service:

foo_service = Diplomat::Service.get('foo', :all)
# => [#<OpenStruct Node="hotel", Address="1.2.3.4", ServiceID="hotel_foo", ServiceName="foo", ServiceTags=["foo"], ServicePort=5432>,#<OpenStruct Node="indigo", Address="1.2.3.5", ServiceID="indigo_foo", ServiceName="foo", ServiceTags=["foo"], ServicePort=5432>]

Or if you want to find services for a particular datacenter

foo_service = Diplomat::Service.get('foo', :all, { :dc => 'My_Datacenter'})
# => [#<OpenStruct Node="hotel", Address="1.2.3.4", ServiceID="hotel_foo", ServiceName="foo", ServiceTags=["foo"], ServicePort=5432>,#<OpenStruct Node="indigo", Address="1.2.3.5", ServiceID="indigo_foo", ServiceName="foo", ServiceTags=["foo"], ServicePort=5432>]

Or if you want to filter services

foo_service = Diplomat::Service.get('foo', :all, { :filter => 'ServiceID == "indigo_foo"'})
# => [#<OpenStruct Node="indigo", Address="1.2.3.5", ServiceID="indigo_foo", ServiceName="foo", ServiceTags=["foo"], ServicePort=5432>]

If you wish to list all the services on consul:

services = Diplomat::Service.get_all
# => #<OpenStruct consul=[], foo=[], bar=[]>

If you wish to list all the services for a specific datacenter:

services = Diplomat::Service.get_all({ :dc => 'My_Datacenter' })
# => #<OpenStruct consul=[], foo=[], bar=[]>

Datacenters

Getting a list of datacenters is quite simple and gives you the option to extract all services out of all accessible datacenters if you need to.

datacenters = Diplomat::Datacenter.get()
# => ["DC1", "DC2"]

Sessions

Creating a session:

sessionid = Diplomat::Session.create({:Node => "server1", :Name => "my-lock"})
# => "fc5ca01a-c317-39ea-05e8-221da00d3a12"

Or destroying a session:

Diplomat::Session.destroy("fc5ca01a-c317-39ea-05e8-221da00d3a12")

Renew a session:

Diplomat::Session.renew(sessionid)

List sessions:

Diplomat::Session.list.each {|session| puts "#{session["ID"]} #{session["Name"]}"}

Locks

Acquire a lock:

sessionid = Diplomat::Session.create({:Node => "server1", :Name => "my-lock"})
lock_acquired = Diplomat::Lock.acquire("/key/to/lock", sessionid)
# => true

Or wait for a lock to be acquired:

sessionid = Diplomat::Session.create({:hostname => "server1", :ipaddress => "4.4.4.4"})
lock_acquired = Diplomat::Lock.wait_to_acquire("/key/to/lock", sessionid)

Release a lock:

Diplomat::Lock.release("/key/to/lock", sessionid )

Events

Fire an event:

Diplomat::Event.fire('do_something', 'payload')

List all events with a certain name received by the local agent:

Diplomat::Event.get_all('do_something')

Get the latest event with a certain name received by the local agent:

Diplomat::Event.get('do_something')

Iterate through the events with a certain name received by the local agent:

events = Enumerator.new do |y|
  ret = {token: :first}
  while ret = begin Diplomat::Event.get('do_something', ret[:token], :reject) rescue nil end
    y.yield(ret[:value])
  end
end

events.each{ |e| puts e }

Status

Returns information about the status of the Consul cluster.

Get the raft leader for the datacenter in which the local consul agent is running

Diplomat::Status.leader()

Get an array of Raft peers for the datacenter in which the agent is running

Diplomat::Status.peers()

Autopilot

Returns information about the autopilot configuration of the Consul cluster

Get the current autopilot configuration

Diplomat::Autopilot.get_configuration()

Get the health status from autopilot

Diplomat::Autopilot.get_health()

Health

Retrieve health of a nod