A Guide for Writing Maintainable Rails Tests

Written by Matt on / Dec 17

This article was originally published at Littlelines.

Do you ever feel like you spend most of your day repairing tests in your Rails app? If you have been building Rails apps as long for as we have, then you know the importance of a robust test suite. Working with a brittle and slow tests can really make the most basic tasks difficult. This is especially true for large Rails apps that have been around for a few years. The good news is that we can fix this with a few helpful tips I've picked up over the years that can keep your test suite running smoothly for the long-run.

First, let's define what maintainable means. It can take on a lot meanings, but let's break it down into three categories:

  1. Tests should be reliable. When we run our tests over and over; whether it's a single test or the entire test suite, we want them to consistently pass or even consistently fail. There's nothing more tedious than tracking down inconsistencies. I've spent countless hours and sometimes days fixing tests that run fine in isolation, but fail when running the complete suite.

  2. Tests should be easy to write. I confess, when I work with a Rails app that has slow or brittle tests, I skip writing new tests all together because I know it's too difficult or too time consuming setting up new tests. So it's critical that our test suite be straightforward enough to dive right in.

  3. Tests should be easy to understand. When tests fail, we should be able to look at the test code and quickly understand why it's failing and how to fix it. The last thing we want to do is waste time staring at code wondering "Why is this failing!".

The Test Environment

Now that we have an understanding of what maintainable tests mean, let's get started with a solid foundation - the test helpers. Here is what our test_helper.rb might look like:

require "minitest/autorun"
require "mocha/setup"
require 'simplecov'
require 'capybara/dsl'
require 'capybara/rails'

SimpleCov.start

module TestHelper
  include Capybara::DSL

  def setup
    DatabaseCleaner.strategy = :truncation
    DatabaseCleaner.clean_with(:truncation)
    DatabaseCleaner.start
  end

  def teardown
    DatabaseCleaner.clean
  end
end

Nice and simple! Let's take a closer at the gems we're using:

Minitest - I like to use MiniTest for writing tests because of its simplicity and it provides a complete set of testing facilities without the noise. Best of all it's shipped with Ruby 1.9.

Mocha - When we absolutely need to mock or stub, I reach for Mocha since it's a good lightweight option. I found Mocha to be the least fussy when it comes to stubbing.

SimpleCov is absolutely essential for measuring our code coverage. Our road to lean and mean tests start by knowing what has already been tested and what has yet to be tested.

DatabaseCleaner To avoid banging our head against bizarre inconsistencies with the test database, we are going to call on the help from DatbaseCleaner, which ensures that we have a clean database state between tests.

Capybara Capybara is an intuitive web browser simulation framework that allows us to test how a real user interacts with our web application. It also comes with a built-in DSL for describing user interactions. We'll use Capybara extensively for our acceptance tests.

Achieving Testing Zen

Now that we have a nice setup for our suite, we're ready to start adding tests. How we approach testing is important if we want to keep our test suite healthy. We want to write the minimal amount of code required to satisfy our test cases. The best way to do this is using the Top-Down approach. This brings us to our first tip.

Tip #1: Start at the top. Start with the acceptance tests and drive our implementation at the user interface level. This can be a great way to prototype our implementation, and starting at the top, we can cover a lot of code with little tests. Once we are satisfied with our acceptance tests, we can fill-in holes with unit tests for complete public interface coverage.

Tip #2: Use helper methods to keep your test DRY. To keep our tests nice and neat, extract common setup scenarios into helper methods. For example, if we know we have to log into the application to test our app, we can setup a handy login method:

def log_in_user(user)
  User.stubs(:authenticate).returns(user)
  visit "/login"

  within("form") do
    fill_in "login", with: user.login
    fill_in "password", with: "anything"
  end
  click_button "Log in"
end

Tip #3: Avoid Copy and Paste. Sometimes it's so easy to copy and paste code from one test to another - especially our setup code. Before you copy and paste, ask yourself "Should I move this code to a helper method instead?".

Tip #4: Don't test what has already been tested. It may sound simple, but chances are you've written tests for code that has already been tested without knowing it. I come across something like this quite often:

describe User do
  it { should have_many(:orders)}
end

What exactly are we testing here? Are we just wanting to be extra sure that we've spelled the has_many :order association correctly in the User model? ActiveRecord associations are well-tested by the Rails test suite, so there is not need to have this in our test suite.

Tip #6: Avoid Excessive use of mocks, stubs, and expectations. Too many mocks and stubs can lead to unexpected results and expectations often serve no benefit at all and almost always lock you to the internal implementation of the thing you're testing, thus making tests brittle. I have seen many projects use excessive mocks and often times it's not testing anything. Prefer to use real data from the database. The downside of this of course, is that our tests may be slower.

Tip #7: Makes tests fast enough. We're not focusing too much on speed. Don't get me wrong, if your tests are slow, you won't be encouraged to write them. So it's important to keep speed in mind. But in the end, the most important thing we want is a reliable, understandable code, even if they're not as fast as we'd like them to be.

Tip #8: Keep context and describe blocks flat. Avoid deeply nested describe contexts. For example:

describe "checking out" do

  describe "add items in shopping cart" do
    before do
      add_item_to_shopping_cart(product)
      click_link 'Checkout'
    end
    ...

    describe "pay with credit card" do
      ...
    end
  end
end

When go too deep in our describe blocks, it's easy to lose sight of the steps taken in the previous describe blocks. To compound the problem, we can't be sure of the database state either. Consider breaking describe blocks into separate tests or even separate files when deep nesting occurs. For our example above, we'll create a brand new test file for paying with credit card:

describe "paying with credit card" do
  before do
    setup_shopping_cart_and_checkout
    fill_in_credit_card_fields
  end

  it 'checks for valid promotion codes' do
    fill_in "promo_code", with: 'does not exists'
    click_link 'Place Order'
    assert page.has_content?("Sorry, invalid promotion code")
  end
end

Tip #9: Only test public side effects. If a method changes the internal state, write your assertions based on the side effects available to the public. Keep test cases to external API - allows us to change internal implementation, while ensuring the consumers of your class still work.

And that is it! By following these tips, you're on your way to achieving a happy and health test suite. If you found this article helpful, you should also checkout:

  1. Practical Guide to User Testing
  2. Rails Testing Pyramid
  3. Minitest Quick Reference

Got any tips to share? Please leave them in the comments.


Minitest Quick Reference

Written by Matt on / Dec 10

UPDATE: I've added a new section on stubbing with MiniTest and a few helpful comments to the code samples.

MiniTest, as the name suggests, is a small and fast unit testing framework. Shipped with Ruby 1.9, MiniTest supports a complete suite of testing capabilities such as TDD, BDD, mocking, and benchmarking.

This quick reference aims to demonstrate MiniTest's main concepts and provide real world examples to get you acquainted quickly. Let's start with MiniTest::Spec.

MiniTest::Spec

Provides RSpec-like matchers and contexts right out of the box.

require 'minitest/autorun'

describe Hipster, "Demonstration of MiniTest" do

  # Runs codes before each expectation
  before do
    @hipster = Hipster.new
  end

  # Runs code after each expectation
  after do
    @hipster.destroy!
  end

  # Define accessors - lazily runs code when it's first used
  let(:hipster) { Hipster.new}
  let(:traits) { ["silly hats", "skinny jeans"] }
  let(:labels) { Array.new }

  # Even lazier accessor - assigns `subject` as the name for us
  # this equivalent to let(:subject) { Hipster.new }
  subject { Hipster.new }

  it "#define" do
    hipster.define.must_equal "you wouldn't understand"
  end

  it "#walk?" do
    skip "I prefer to skip"
  end

  describe "when asked about the font" do
    it "should be helvetica" do
      @hipster.preferred_font.must_equal "helvetica"
    end
  end

  describe "when asked about mainstream" do
    it "won't be mainstream" do
      @hipster.mainstream?.wont_equal true
    end
  end
end

Matchers (must | wont)

In most cases you can switch between must for positive expectations and wont for negative expectations.

Assertion Examples
must_be labels.size.must_be :==, 0
must_be_close_to traits.size.must_be_close_to 1,1
must_be_empty labels.must_be_empty
must_be_instance_of hipster.must_be_instance_of Hipster
must_be_kind_of labels.must_be_kind_of Enumerable
must_be_nil labels.first.must_be_nil
must_be_same_as traits.must_be_same_as traits
must_be_silent proc { "no stdout or stderr" }.must_be_silent
must_be_within_epsilon traits.size.must_be_within_epsilon 1,1
must_equal traits.size.must_equal 2
must_include traits.must_include "skinny jeans"
must_match traits.first.must_match /silly/
must_output proc { print "#{traits.size}!" }.must_output "2!"
must_respond_to traits.must_respond_to :count
must_raise proc { traits.foo }.must_raise NoMethodError
must_send traits.must_send [traits, :values_at, 0]
must_throw proc { throw Exception if traits.any? }.must_throw Exception

MiniTest::Unit::TestCase

Provides a rich set of assertions to make your tests clean and readable.

require 'minitest/autorun'

class TestHipster < MiniTest::Unit::TestCase
  def setup
    @hipster = Hipster.new
    @labels  = Array.new
    @traits  = ["silly hats", "skinny jeans"]
  end

  def teardown
    @hipster.destroy!
  end

  def test_for_helvetica_font
    assert_equal "helvetica!", @hipster.preferred_font
  end

  def test_not_mainstream
    refute @hipster.mainstream?
  end
end

Assertions (assert | refute)

Toggle between assert for positive assertions and refute for negative assertions.

Assertion Example
assert assert @traits.any?, "empty subjects"
assert_empty assert_empty @labels
assert_equal assert_equal 2, @traits.size
assert_in_delta assert_in_delta @traits.size, 1,1
assert_in_epsilon assert_in_epsilon @traits.size, 1, 1
assert_includes assert_includes @traits, "skinny jeans"
assert_instance_of assert_instance_of Hipster, @hipster
assert_kind_of assert_kind_of Enumerable, @labels
assert_match assert_match @traits.first, /silly/
assert_nil assert_nil @labels.first
assert_operator assert_operator @labels.size, :== , 0
assert_output assert_output("Size: 2") { print "Size: #{@traits.size}"}
assert_raises assert_raises(NoMethodError) { @traits.foo }
assert_respond_to assert_respond_to @traits, :count
assert_same assert_same @traits, @traits, "It's the same object silly"
assert_send assert_send [@traits, :values_at, 0]
assert_silent assert_silent { "no stdout or stderr" }
assert_throws assert_throws(Exception,'is empty') {throw Exception if @traits.any?}

MiniTest#stub

Minitest provides a simple stub method we can use to return a pre-determined value.

require 'minitest/autorun'

describe Hipster, "Demonstrates stubbing with Minitest" do

  let(:hipster) { Hipster.new }

  it "trendy if time is now" do
    assert hipster.trendy? DateTime.now
  end

  it "it is NOT trendy if 2 weeks has past" do
    DateTime.stub :now, (Date.today.to_date - 14) do
      refute hipster.trendy? DateTime.now
    end
  end
end

MiniTest::Mock

A simple and clean mock system. There two essential methods at our disposal: expect and verify.

require 'minitest/autorun'

# Make all of our Twitter updates hip
class Twipster
  def initialize(twitter)
    @twitter = twitter # A Twitter API client
  end

  def tweet(message)
    @twitter.update("#{message} #lolhipster")
  end
end

# Uses Mock#expect and Mock#verify
describe Twipster, "Make every tweet a hipster tweet." do
  before do
    @twitter  = MiniTest::Mock.new # Mock our Twitter API client
  end

  let(:twipster) { Twipster.new(@twitter) }
  let(:message) { "Skyrim? Too mainstream."}

  it "should append a #lolhipster hashtag and update Twitter with our status" do
    @twitter.expect :update, true, ["#{message} #lolhipster"]
    @twipster.tweet(message)

    assert @twitter.verify # verifies tweet and hashtag was passed to `@twitter.update`
  end
end

Resources

  1. MiniTest on Github
  2. MiniTest Rdoc
  3. Using MiniTest::Spec with Rails
  4. Ruby Inside: A MiniTest::Spec Tutorial: Elegant Spec-Style Testing That Comes With Ruby

I hope you found this quick guide valuable. Please let me know if you'd like to see anything else included and feel free to ask questions or give feedback in the comments section.


Ruby Blocks as Dynamic Callbacks

Written by Matt on / Nov 27

Callbacks are a great technique for achieving simplicity and flexibility. Simply put, a callback is a block of code passed as an argument to a method. In Ruby, code blocks are everywhere and Ruby makes it trivial to pass a block of code to methods. For example:

def foo(bar, &block)
  callback = block
  callback.call(bar)
end

foo(5) {|x| x * x} # => 25

But what do we do when a method needs two blocks of code or more? Consider the classic case where we want a method to execute a block of code if an action succeeds or call different code if an action fails.

In this article, I will demonstrate how we can pass multiple blocks to a method and with some metaprogramming, we can achieve a dynamic callback mechanism with just a few lines of code.

Let's add a method called callback to the Proc class:

class Proc
  def callback(callable, *args)
    self === Class.new do
      method_name = callable.to_sym
      define_method(method_name) { |&block| block.nil? ? true : block.call(*args) }
      define_method("#{method_name}?") { true }
      def method_missing(method_name, *args, &block) false; end
    end.new
  end
end

That's it! The above Proc#callback method simply yields an anonymous class with methods defined to handle our callbacks. This allows for the capability of creating and storing dynamic callbacks, which can later be looked up and executed as needed.

Notice anything unusual? We're using the === operand to invoke the block. Proc#=== is an alias for Proc.call. Anything on the right side of === acts as the proc's parameter. Normally, this is to allow a proc object to be a target of a when clause in case statements, but we're using it as a super simple way of invoking our anonymous class.

Let’s try it with something useful. Let’s say we’re writing something which needs to happen in an all-or-nothing, atomic fashion. Either the whole thing works, or none of it does. A simple case is tweeting:

def tweet(message, &block)
  Twitter.update(message)
  block.callback :success
rescue => e
  block.callback :failure, e.message
end

The tweet method accepts a message string and &block parameters. We call callback on the block and give it a name. Any name will work :success, :error, :fail!, whatever. In addition, we can pass arguments to the blocks (more on that later). Now we can provide a status if the tweet was successful or not:

tweet "Ruby methods with multiple blocks. #lolruby" do |on|
  on.success do
    puts "Tweet successful!"
  end
  on.failure do |status|
    puts "Error: #{status}"
  end
end

The advantage here is that we define our own mini DSL. We don't need to worry about passing too many or unexpected blocks. We could have easily said where.success or on.error or update.fail!. Also note the on.failure block includes a status parameter - this contains the exception message captured in the tweet method above. So if Twitter was down for whatever reason, the on.failure block would be invoked and printed 'Error: Twitter is down or being upgraded'.

Bonus: In addition to wrapping code in blocks, our Proc#callback method defines boolean style methods. So we could have call the tweet method like this if we wanted to:

tweet "Ruby methods with multiple blocks. #lolruby" do |update|
  puts "Tweet successful!" if update.success?
  puts "Sorry, something went wrong." if update.failure?
end

Put the Proc#callback method in a utility library and your code will look neat and tidy.

As always, I welcome your thoughts and feedback. Let me know what you think of the techniques shown here, or share your own favorite code block tricks.