Pragmatic Concurrency With Ruby

I'm coming from a parallel computation, distributed systems background by education, and have relatively strong foundations in infrastructural concurrent/parallel libraries and products that I've built and maintained over the years, on both the JVM and .NET.

Recently, I've dedicated more and more time building and deploying real concurrent projects with Ruby using JRuby, as opposed to developing with Ruby (MRI) with concurrency the way it is (process-level and GIL thread-level). I'd like to share some of that with you.

Administrative note<<EOF:

This may come as a lengthy information-packed read. You can put the blame on me for this one, because I wanted to increase the value for the reader as much as possible, and pack something that could have been a lengthy book, into a single highly concentrated no-bullshit article.

As an experiment, I also put most of the example code in a repository including the source of this article. Please feel free to fork and apply contributions of any kind, I'll gladly accept pull requests.

Github repo: https://github.com/jondot/pcwr

EOF

Concurrency is Awesome!

Remember those old 8-bit games you used to play as a child?. In a hindsight - you know its awesome, but if you're a gamer or just a casual gamer, and you're forced to play it today, the graphics will feel bad.

This is because it’s a detail thing; just like childhood computer games, as time passes, it seems like your brain doesn't care (or forgets) the proper details.

So given that one is an MRI Ruby developer, her mindset would be that concurrency just works, and it is easy and awesome. But you might be right guessing that due to the level of cynicism going around here - it isn't the end of it.

The MRI Ruby GIL is gracefully keeping some details away from you: yes things are running in parallel with the help of properly built I/O libraries (for example: historically, the MySQL gem was initially not doing it properly, which meant your thread would block on I/O), but surely, code isn't running in parallel. It's just like what your brain did when it covered up for those horrific 8-bit graphics that you were sure are still awesome.

<!-- more -->

For real concurrency that isn't soft padded by the GIL? if you're not armed with the proper mental and practical tools, there's a good chance you'll quickly find yourself in front of nasty race conditions. The human mind only goes so much in order to reason about parallel events.

And when you're stuck rummaging through tons of stack traces from memory dumps you just took off production? you'll have to CSI yourself out of the situation, at which point I bet you'll beg to have a single process, 0-concurrency thing running instead.

MRI Ruby helps you. It can guarantee that you don't have real concurrency, at the price of not running code in a really parallel way.

And that's fine. But today, lets walk through real concurrency so that you'll have the proper intuition and background when you need to break out of these limitations.

Why?

Several years ago, I worked on an enterprise-grade software in the [DFM](http://en.wikipedia.org/wiki/Design_for_manufacturability_(PCB) / electronics field at Mentor Graphics. There was one product, which was able to optimize the kind of machines that fabricate the components of the phones, laptops and electronics that you use day to day and all aspects of their manufacturing operation, so that they were used to the maximum possible capacity. It did so by modeling the manufacturing pipeline in ways that humans never could.

In this field, a machine not working at its 100% capacity is directly translated to money lost. And worse, if a bug in the optimizing software caused a few hours of downtime, that's plain catastrophic.

By an analogy, CPUs were made to burn cycles. If you have a machine that doesn't work all of its cores, all on 100% CPU and in the first place you need more performance out of your application, then concurrency is one of the solutions.

The world is heading towards a distributed and concurrent processing model, that would allow squeezing every bit of juice out of machines, using any number of machines.

A great starting point is to allow your Ruby code to utilize all that it can within the framework of itself, and by that, to be in a better position to take part in this move as well.

State of The Union

Having a shared state, visible to everyone, and without a way of telling who's using it, is insane. It's more insane when you throw in several threads to mutate it.

Putting Ruby aside for a moment, one of the most agonizing things for me when reviewing Java or .Net code, is to see a developer using static variables.

If you haven't yet the chance to use a statically typed language like Java or C#, then the term static is used in the sense that there is one instance of the thing, well defined as 'static' by the hosting language, and is visible implicitly, i.e. no one is explicitly declaring it in any interface, or explicitly passing it to other objects (as a simplistic example: a global variable).

Since these languages are statically typed, there's always a feeling that its OK to prematurely optimize bits, because we're on a statically typed language and performance is always expected of us; we should always be on guard. This creates a reality where developers often prematurely optimize things.

Back to Ruby. Going through popular frameworks and gems, I can't but admit that Ruby has grown to be 'static happy'. But I think its for an entirely different reason -- which is: threads were never there to challenge the concept of using and mutating global state.

In Ruby, that one static always-visible container, often sneaks out. In a hindsight you'll note that some times it simply serves as a poor man's Inversion-of-Control (“IoC”) - and that's cool; other times, it is misused as a big ball of mud, often mutated all along the running code by many different parts of your applications. In any case, on real concurrent platforms such as JRuby its still plain dangerous.

That's why even on MRI, I cringe every time I have to use a static variable in the framework/library I'm basing my code on, and every time I'm using an object that I know will use a static variable, because as you might already know, that shit is transitive:

module MyLibrary
  # nasty static variable
  BigBallOfMud = { :woofed => 0 }
end

class Puppy
  def woof!
    puts "woof!"
    # for book keeping
    MyLibrary::BigBallOfMud[:woofed] += 1
  end
end

pup = Puppy.new

# eventhough pup isn't static and is cute, woof! is still pretty nasty because
# it accesses a static variable.
pup.woof!

How dangerous is it? look:

100.times do
 Thread.new do
   pup.woof!
 end
end

sleep(5)
puts MyLibrary::BigBallOfMud[:woofed]

I have removed the puts from woof! in order to position thread contention in a better way, and running it on JRuby gives:

$ ruby test.rb
97

And the result varies from run to run due to the timing of your threads. MRI Ruby on the other hand correctly and reliably produces 100. The way this happens is due to MRI inlining, treating += not as a separate access and assignment - but it serves well to visualize the point (thanks @thedarkone for giving an indepth explanation of this).

Problem is, I have the feeling that everyone around me are in a non-concurrent world, and are accustomed to keeping and using (sharing) global static variables while never flipping that "what if another thread gets here" switch that kills everyone's party.

Thread Safety

The first step into the concurrent world, is to use a real concurrent platform, for us Rubyists, it is indisputably JRuby. Not to mention, the JVM is still very strong, and really great for server-side work.

The next step is to care and worry about thread safety. If you're using a 3rd party ruby gem, or a Java library, you should be aware that there's a chance it is not thread safe, and once more than one of your threads hit it at the same time it'll blow up, and just to further emphasize on the previous discussion - wouldn't have blown up if your code was running on MRI.

Sure, you can blow things up with Ruby threads; doing shared I/O for example but this is besides the point for now: I'm talking about code running in parallel, mutating shared state.

Lets take a look at this simplistic statistics metrics server written in Sinatra.

require 'sinatra'
require 'json'


class StatsApp < Sinatra::Base
  configure :production, :development do
    set :stats, {}
    puts "configured app"
  end

  post '/stats/:product/:metric' do
    settings.stats[params[:product]] ||= {}
    settings.stats[params[:product]][params[:metric]] ||= 0
    settings.stats[params[:product]][params[:metric]] += 1
    "ok"
  end

  get '/stats/:product' do
    settings.stats[params[:product]].to_json
  end
end

Note that we idiomatically initialize null references when we encounter a new metric and a new product. Let’s run it with MRI and thin:

$ ab -n 20000 -c 100 -p /dev/null http://localhost:9292/stats/foo/mongodb.read
  ... snip ...
  Complete requests:      20000
  ... snip ...
  Requests per second:    683.95 [#/sec] (mean)
  ... snip ...

And now:

$ curl http://localhost:9292/stats/foo
   {"mongodb.read":200}

Not bad for a low-resource VM. Le