Squash: A squarish bug spray

An open-source project from Square

Squash is a collection of tools that help engineers find and kill bugs in their code by automatically collecting, collating and analyzing run time exceptions. Squash consists of the following components:

• Client libraries: Client libraries for different systems (Ruby, Ruby on Rails, Cocoa with Objective-C, etc.) catch and record errors when they occur, then send them to the API endpoint. Client libraries can be found under the SquareSquash organization.
• Front-end: This website displays Bug information and helps the engineer find the root cause for a Bug, and fix it. It also lets engineers manage, assign, and comment on Bugs, as well as view statistics about the Bug.
• API endpoints: These routes (part of the front-end app) receive exception notifications from the client libraries and process them.

This project is the front-end and the API.

Pull requests are more than welcome; please check out CONTRIBUTING.md for details.

How to Install

To get started, run the bin/setup file. This script will ask you a series of questions and generate a preliminary configuration for your site install. When the script is completed, you can run git status to see what files it changed, and refine your configuration from there.

Once the script is complete, Squash should run for most typical development environments. Simply run rails s and visit the site in your Web browser. You should be able to start using it immediately. You can also verify correctness by running rspec spec.

Configuring and deploying the production instance is entirely up to you and your particular production environment.

Additional configuration options can be found in the following locations:

• config/application.rb
• config/environments/*.rb
• config/environments/*/*.yml

If you don't see what you're looking for in any of those files, you'll probably have to change the code to make it work. Don't be afraid -- the code is thoroughly documented and should (hopefully) be very accessible.

Requirements

Squash requires the following:

• Ruby 1.9.2 or newer (JRuby with --1.9 is supported)
• Multithreading support (see the next section)
• PostgreSQL 9.1 or newer
• The Bundler gem
• Git 1.7 or newer

Notes on some of the gem and library choices

Why do you specifically require PostgreSQL? Squash uses a lot of PostgreSQL-specific features to make efficient use of the database and maintain referential integrity, such as:

• foreign key constraints,
• triggered cached counters,
• check constraints, and
• semantic indexes for text-based search.

If PostgreSQL is out of the question, some of these features can be ported to other RDBMSes; simply edit the InitialSchema migration and update the SQL as necessary. If portability is required, a lot of these features can be reimplemented in the Rails layer (e.g., cached counters), at the risk of degraded referential integrity. (If you are reimplementing the cached counters in Ruby, be sure to modify the config/initializers/active_record.rb file as appropriate.)

If you do successfully port Squash to another RDBMS, let me know. I'd be happy to take your changes.

Why do you bundle an edge version of Rails? The 3-2-stable branch of Ruby on Rails includes some changes to Active Record that are required by Squash's multithreaded concurrency model (see next question). In particular, that version of Active Record includes crucial changes to the connection pool and connection reopening logic. If you set the background_runner option in the concurrency.yml file to something other than Multithread, the Gemfile will automatically drop Rails back down to the latest release version.

Why don't you use my favorite backgrounding library? Squash was originally built for Square, which runs all its services on JRuby. Using threads is very efficient in JRuby, and avoids the overhead of having to deploy both a website and workers.

If you are running Squash in a non-thread-safe (or multithreading-unfriendly) environment, you can use Sidekiq or Resque instead. If you want to use some other backgrounding library, you can easily write your own adapter. All threaded code is invoked using {BackgroundRunner.run}, which then uses the settings in the concurrency.yml Configoro file to invoke the correct module under lib/background_runner. The default is to use {BackgroundRunner::Multithread}, which uses the {Multithread} module to execute the task in its own thread. You can edit the YAML file and switch the background runner to Resque, or implement your own BackgroundRunner module. See the {BackgroundRunner} documentation for more details.

If you do this successfully and wish to save future Squash users the effort, feel free to turn your changes into a pull request.

Why aren't you using RedCarpet? As mentioned above, Squash was originally built to run under JRuby. RedCarpet has compiled C extensions; Kramdown is pure Ruby.

Why do you require a recent version of Git? Squash uses the git clone --mirror command to create local mirrors of client projects' Git repositories.

Why don't you have integration tests or acceptance tests? To be 100% honest, lack of familiarity on these things in Ruby/Rails. Happy for any help people want to extend towards this goal.

Why are you using Erector? I like Erector.

Documentation

Comprehensive documentation is written in YARD- and Markdown-formatted comments throughout the source. To view this documentation as an HTML site, run rake yard.

CoffeeScript libraries are documented using the YARD format as well, but YARD does not as yet recognize them as documentable files. A .codoopts file is included in case you wish to use Codo to generate the CoffeeScript docs, but as of now Codo does not recognize the ERb files, and does not use the full set of Markdown syntax features used in the documentation.

Project Overview

Views

This is a pretty typical Rails website, save for the views, which are written using Erector. The views forgo the traditional Rails concepts of partials and templates in favor of analogous OOP concepts more familiar to software developers: methods and inheritance. All views inherit from an abstract Erector widget which provides layout; and all views have their content split into multiple private methods.

In addition to the usual helpers (in app/helpers), there are view mixins under app/views/additions that simplify view coding.

Embedded code snippets are all rendered using the {CommitsController#context} action. This action loads the appropriate file and revision from the Git repository and returns a snippet plus the name of the SyntaxHighlighter brush to use. The brush is determined from the file name/extension; the mapping can be found in data/brushes.yml.

This view behavior is provided from a JavaScript library file in lib/assets/javascripts. There are many similar helper classes in there; they are documented but the documentation is not recognized by YARD and so is not included in this documentation set.

JavaScript files are organized into four possible locations:

• Third-party JavaScript libraries are in vendor/assets/javascripts and loaded in the application.js manifest.
• JavaScript modules or helpers that are not specific to a particular page or site area are in lib/assets/javascripts and also loaded in application.js.
• JavaScript modules or helpers specific to a particular area of the site are in app/assets/javascripts and also loaded in application.js.
• Small JavaScript snippets, glue code, or other code intended to add dynamic behavior to a specific page is in a .js file named the same as, and placed alongside, the .html.rb view file. For example, if app/views/projects/new.html.rb needed a bit of JS glue code, it would be placed in app/views/projects/new.js. This code is placed in a <SCRIPT> tag at the end of the view by the {Views::Layouts::Application#inline_javascript} method.

CSS files are similarly organized:

• Third-party CSS files are in vendor/assets/stylesheets and loaded in the application.css manifest.
• CSS styles or helpers global to the entire website are in lib/assets/stylesheets and also loaded in application.css.
• CSS styles specific to a single page or a related group of pages are placed in app/assets/stylesheets and also loaded in application.css. Each <BODY> tag is given a class name equal to the controller name, and an ID equal to the controller and action name separated with a dash. For instance, the projects/new action's body would be <body class=projects id=projects-new>.

Controllers

For information about requests and responses, see {ApplicationController}.

Models

Models make extensive use of advanced PostgreSQL features for efficiency and convenience. Cached counters are updated using triggers and rules, foreign key constraints and hooks are enforced at the database level, and validations are backed up by corresponding CHECK triggers. This helps ensure referential and data integrity even in situations where Rails fails, or outside of the Rails stack. See the various migrations to learn more about the triggers, rules, and constraints being used.

Observers are used for more high-level triggers, such as creating {Event Events} at the appropriate times, or sending emails. See the classes in app/models/observers for more.

Models also use the HasMetadataColumn gem to reduce their width and incorporate schemaless data. Most models have a JSON-formatted metadata column to which new information can be added or removed without having to create new migrations.

Tasks

Various Rake tasks