Sshkit

SSHKit is a toolkit for running commands in a structured way on one or more servers.

Example

• Connect to 2 servers
• Execute commands as deploy user with RAILS_ENV=production
• Execute commands in serial (default is :parallel)

require 'sshkit'
require 'sshkit/dsl'
include SSHKit::DSL

on ["1.example.com", "2.example.com"], in: :sequence do |host|
  puts "Now executing on #{host}"
  within "/opt/sites/example.com" do
    as :deploy  do
      with RAILS_ENV: 'production' do
        execute :rake, "assets:precompile"
        execute :rails, "runner", "S3::Sync.notify"
      end
    end
  end
end

Many other examples are in EXAMPLES.md.

Basic usage

The on() method is used to specify the backends on which you'd like to run the commands. You can pass one or more hosts as parameters; this runs commands via SSH. Alternatively you can pass :local to run commands locally. By default SSKit will run the commands on all hosts in parallel.

Running commands

All backends support the execute(*args), test(*args) & capture(*args) methods for executing a command. You can call any of these methods in the context of an on() block.

Note: In SSHKit, the first parameter of the execute / test / capture methods has a special significance. If the first parameter isn't a Symbol, SSHKit assumes that you want to execute the raw command and the as / within / with methods, SSHKit.config.umask and the comand map have no effect.

Typically, you would pass a Symbol for the command name and it's args as follows:

on '1.example.com' do
  if test("[ -f somefile.txt ]")
    execute(:cp, 'somefile.txt', 'somewhere_else.txt')
  end
  ls_output = capture(:ls, '-l')
end

By default the capture methods strips whitespace. If you need to preserve whitespace you can pass the strip: false option: capture(:ls, '-l', strip: false)

Transferring files

All backends also support the upload! and download! methods for transferring files. For the remote backend, the file is transferred with scp by default, but sftp is also supported. See EXAMPLES.md for details.

on '1.example.com' do
  upload! 'some_local_file.txt', '/home/some_user/somewhere'
  download! '/home/some_user/some_remote_file.txt', 'somewhere_local', log_percent: 25
end

Users, working directories, environment variables and umask

When running commands, you can tell SSHKit to set up the context for those commands using the following methods:

as(user: 'un', group: 'grp') { execute('cmd') } # Executes sudo -u un -- sh -c 'sg grp cmd'
within('/somedir') { execute('cmd') }           # Executes cd /somedir && cmd
with(env_var: 'value') { execute('cmd') }       # Executes ENV_VAR=value cmd
SSHKit.config.umask = '077'                     # All commands are executed with umask 077 && cmd

The as() / within() / with() are nestable in any order, repeatable, and stackable.

When used inside a block in this way, as() and within() will guard the block they are given with a check.

In the case of within(), an error-raising check will be made that the directory exists; for as() a simple call to sudo -u <user> -- sh -c <command>' wrapped in a check for success, raising an error if unsuccessful.

The directory check is implemented like this:

if test ! -d <directory>; then echo "Directory doesn't exist" 2>&1; false; fi

And the user switching test is implemented like this:

if ! sudo -u <user> whoami > /dev/null; then echo "Can't switch user" 2>&1; false; fi

According to the defaults, any command that exits with a status other than 0 raises an error (this can be changed). The body of the message is whatever was written to stdout by the process. The 1>&2 redirects the standard output of echo to the standard error channel, so that it's available as the body of the raised error.

Helpers such as runner() and rake() which expand to execute(:rails, "runner", ...) and execute(:rake, ...) are convenience helpers for Ruby, and Rails based apps.

Verbosity / Silence

• raise verbosity of a command: execute "echo DEAD", verbosity: :ERROR
• hide a command from output: execute "echo HIDDEN", verbosity: :DEBUG

Parallel

Notice on the on() call the in: :sequence option, the following will do what you might expect:

on(in: :parallel) { ... }
on(in: :sequence, wait: 5) { ... }
on(in: :groups, limit: 2, wait: 5) { ... }

The default is to run in: :parallel which has no limit. If you have 400 servers, this might be a problem and you might better look at changing that to run in groups, or sequence.

Groups were designed in this case to relieve problems (mass Git checkouts) where you rely on a contested resource that you don't want to DDOS by hitting it too hard.

Sequential runs were intended to be used for rolling restarts, amongst other similar use-cases.

The default runner can be set with the SSHKit.config.default_runner option. For example:

SSHKit.config.default_runner = :parallel
SSHKit.config.default_runner = :sequence
SSHKit.config.default_runner = :groups
SSHKit.config.default_runner = MyRunner # A custom runner

If more control over the default runner is needed, the SSHKit.config.default_runner_config can be set.

# Set the runner and then the config for the runner
SSHKit.config.default_runner = :sequence
SSHKit.config.default_runner_config = { wait: 5 }

# Or just set everything once
SSHKit.config.default_runner_config = { in: :sequence, wait: 5 }

Synchronisation

The on() block is the unit of synchronisation, one on() block will wait for all servers to complete before it returns.

For example:

all_servers = %w{one.example.com two.example.com three.example.com}
site_dir    = '/opt/sites/example.com'

# Let's simulate a backup task, assuming that some servers take longer
# then others to complete
on all_servers do |host|
  within site_dir do
    execute :tar, '-czf', "backup-#{host.hostname}.tar.gz", 'current'
    # Will run: "/usr/bin/env tar -czf backup-one.example.com.tar.gz current"
  end
end

# Now we can do something with those backups, safe in the knowledge that
# they will all exist (all tar commands exited with a success status, or
# that we will have raised an exception if one of them failed.
on all_servers do |host|
  within site_dir do
    backup_filename = "backup-#{host.hostname}.tar.gz"
    target_filename = "backups/#{Time.now.utc.iso8601}/#{host.hostname}.tar.gz"
    puts capture(:s3cmd, 'put', backup_filename, target_filename)
  end
end

The Command Map

It's often a problem that programmatic SSH sessions don't have the same environment variables as interactive sessions.

A problem often arises when calling out to executables expected to be on the $PATH. Under conditions without dotfiles or other environmental configuration, $PATH may not be set as expected, and thus executables are not found where expected.

To try and solve this there is the with() helper which takes a hash of variables and makes them available to the environment.

with path: '/usr/local/bin/rbenv/shims:$PATH' do
  execute :ruby, '--version'
end

Will execute:

( PATH=/usr/local/bin/rbenv/shims:$PATH /usr/bin/env ruby --version )

By contrast, the following won't modify the command at all:

with path: '/usr/local/bin/rbenv/shims:$PATH' do
  execute 'ruby --version'
end

Will execute, without mapping the environmental variables, or querying the command map:

ruby --version

(This behaviour is sometimes considered confusing, but it has mostly to do with shell escaping: in the case of whitespace in your command, or newlines, we have no way of reliably composing a correct shell command from the input given.)

Often more preferable is to use the command map.

The command map is used by default when instantiating a Command object

The command map exists on the configuration object, and in principle is quite simple, it's a Hash structure with a default key factory block specified, for example:

puts SSHKit.config.command_map[:ruby]
# => /usr/bin/env ruby

To make clear the environment is being deferred to, the /usr/bin/env prefix is applied to all commands. Although this is what happens anyway when one would simply attempt to execute ruby, making it explicit hopefully leads people to explore the documentation.

One can override the hash map for individual commands:

SSHKit.config.command_map[:rake] = "/usr/local/rbenv/shims/rake"
puts SSHKit.config.command_map[:rake]
# => /usr/local/rbenv/shims/rake

Another opportunity is to add command prefixes:

SSHKit.config.command_map.prefix[:rake].push("bundle exec")
puts SSHKit.config.command_map[:rake]
# => bundle exec rake

SSHKit.config.command_map.prefix[:rake].unshift("/usr/local/rbenv/bin exec")
puts SSHKit.config.command_map[:rake]
# => /usr/local/rbenv/bin exec bundle exec rake

One can also override the command map completely, this may not be wise, but it would be possible, for example:

SSHKit.config.command_map = Hash.new do |hash, command|
  hash[command] = "/usr/local/rbenv/shims/#{command}"
end

This would effectively make it impossible to call any commands which didn't provide an executable in that directory, but in some cases that might be desirable.

Note: All keys should be symbolised, as the Command object will symbolize it's first argument before attempting to find it in the command map.

Interactive commands

(Added in version 1.8.0)

By default