Meta Bates

“Testing is painful.”

“Testing is hard.”

“Testing is complicated.”

“Testing is not fun.”

I hear those sorts of things all the time when I talk to people about testing. I agree that sometimes testing can be all of those things, but if you choose the right tools, the tools that best suite you, testing doesn’t have to be. Let me give you an example of what I’m talking about, how choosing the right tools can make a huge impact on how you feel about testing.

When working for a client recently I came across the need for end to end integration testing. I needed to test, amongst other things, the flow of a user registering through the application in a few different ways. Because registration behaves differently based on where you come from and where you want to go, I needed a good way to test that entire flow, so simple unit and functional tests just were not going to cut it.

In the Ruby community there is a big push to use a testing framework called, Cucumber. Cucumber is a behavior driven development (BDD) tool that let’s you write user stories in plain English. Those stories then get translated into Ruby code that tests those stories against your application. Because of it’s popularity, and some of it’s quite amazing features, this was my first stop on the path to integration testing bliss.

Let me give you an example of a Cucumber script:

Feature: Registration
  In order to use My Great Application
  As a user
  I want to be able to register
  
  Scenario: 'Standard Registration'
    Given I am not currently logged in
    When I am on the signup page
    Then I should see "Sign Up"
    And I fill in "Name (required)" with "Mickey Dolenz"
    And I fill in "Email (required)" with "mickey@monkees.com"
    And I fill in "Password (required)" with "password"
    And I fill in "Password confirmation" with "password"
    And I press "Register"
    Then I should see "Sign Up - Confirm Your Account"
    Then I should be on the registration thank you page
    Then "mickey@monkees.com" should receive an email
    When I open the email
    Then I should see "Confirm my account" in the email body
    When I follow "Confirm my account" in the email
    Then I should be on the welcome page
    And I should see "Welcome to the Great Application"
    
  Scenario: 'Accepting an invitation'
    Given I am not currently logged in
    And the "Boys and Girls Club" invites "mickey@monkees.com" to join
    Then "mickey@monkees.com" should receive an email
    When I open the email
    Then I should see "Accept Invitation" in the email body
    When I follow "Accept Invitation" in the email
    Then I should be on the signup page
    Then I should see "Sign Up"
    And I fill in "Name (required)" with "Mickey Dolenz"
    And the "Email (required)" field should contain "mickey@monkees.com"
    And I fill in "Password (required)" with "password"
    And I fill in "Password confirmation" with "password"
    And I press "Register"
    Then the account "mickey@monkees.com" should be "activated"
    Then I should be on the accept/decline invitation page
    And I should see "Join the Boys and Girls Club"

That script tests the user registration flow through an application in a couple of different ways, first through ‘standard’ registration, and then through being invited to join. Now, the beauty of Cucumber is that these scripts are ‘human’ readable. Your product manager, or other stake holders, should be able to write these scripts themselves, and you, the developer, should be able to just plug them in and code until those scripts pass.

Unfortunately, while that sounds like a little slice of Heaven, the reality is far from it in practice. First, getting stake holders to actually write these ‘stories’, as their typically called, is a tough chore to begin with. If they do write them, they’re typically not going to be ‘plug and play’. Why? Well, when Cucumber reads these scripts it goes line by line and tries to find some code that matches the regular expression of that line and then execute it. If it doesn’t find matching code, then it fails. That means that your stake holders need to write these scripts in a very particular way or developers need to sit down and massage those stories to fit the correct regular expression.

Now, let me just take this opportunity to say that this is not a post about how much I hate Cucumber, in fact I think Cucumber is a pretty amazing piece of software, and does in fact have a lot of great uses. Instead, what I’m talking about it is how Cucumber turned out not to be the right tool for the job for me on a recent project.

So why wasn’t Cucumber the right tool for the job? Great question, glad you asked. Cucumber turned out not to be the right tool for a few reasons. The biggest of which was that I was the one who was writing the user stories. The stake holders had no desire to write these stories, which meant I had to write them. The I had to write the ‘steps’ that back each line of the script. In all fairness, Cucumber does give you some great steps right out of the box. After some fiddling I finally got the Cucumber scripts up and running and testing my work flow. But I definitely ran into some issues.

Because Cucumber isn’t pure Ruby I had a hard time doing something as simple as just printing out the request’s body and headers without having to write a step that did just that, then add that step to my story, etc… It’s overall fiddlyness and non-intuitive way of doing things caused me a lot of grief and time. And, most importantly, I wasn’t really getting the big benefit of Cucumber, stake holder’s writing the stories. So I was doing all this work and not getting the benefits of Cucumber.

So what did I do? I turned to a library called Steak. Steak allows you to write integration tests using pure Ruby and integrates directly in with RSpec, my preferred testing framework. With Steak I was able to write my integration tests in just a few minutes.

require File.expand_path(File.dirname(__FILE__) + '/acceptance_helper')

feature "Registration Steak", %q{
  In order to use My Great Application
  As a user
  I want to be able to register
} do

  scenario "register throught the standard registration process" do
    visit signup_path
    
    expect {
      fill_in_registration(:name => 'Mickey Dolenz', :email => 'mickey@monkees.com')
    }.to change(delivered_emails, :size).by(1)
    
    user = User.find_by_email('mickey@monkees.com')
    user.should be_unverified
    
    page.should have_content('You have signed up successfully. However, we could not sign you in because your account is unconfirmed.')
    current_path.should == root_path
    
    visit signin_path
    fill_in 'user_email', :with => 'mickey@monkees.com'
    fill_in 'user_password', :with => 'password'
    click_button 'Sign In'
    
    page.should have_content('You have to confirm your account before continuing.')
    current_path.should == new_user_session_path

    click_email_link_matching(/users\/confirmation/, delivered_emails.first)

    current_path.should == welcome_users_path
    page.should have_content('Welcome to the Great Application')
    user.reload
    user.should be_activated
  end
  
  scenario "register from an invitation" do
    mark = users(:mark)
    organization = mark.organizations.first
    signin(mark)
    
    visit new_organization_invitation_path(organization)
    expect {
      expect {
        fill_in 'organization_invitation_worker_emails', :with => 'mickey@monkees.com'
        click_button 'Send Invites'
        DJ.first.invoke_job
      }.to change(delivered_emails, :size).by(1)
    }.to change(organization.invitations, :count).by(1)
    
    invitation = organization.invitations.first
    
    signout
    
    click_first_link_in_email(delivered_emails.first)
    
    current_path.should == signup_path

    fill_in_registration(:name => 'Mickey Dolenz')
    
    current_path.should == organization_invitation_path(organization, invitation)
    
    user = User.find_by_email('mickey@monkees.com')
    user.should be_activated
    
    expect {
      click_on 'Get Started Now!'
    }.to change(user.organizations, :count).by(1)
    
    current_path.should == organization_campaign_path(organization, organization.campaigns.first)
  end
  
end

While my Steak scripts a bit more wordy and are definitely not ‘human’ readable and editable by stakeholders, they did achieve my goal of allowing me to write integration tests quickly.

So here you see I picked a very powerful tool, that has a lot of great benefits, Cucumber, but I picked it for the wrong reasons. I picked it because it was popular, and not because it would help me achieve my goals. If my goals where to have stakeholders write the stories and hand them off to development, than it would’ve been a better choice. But in the end my goal was to write integration tests and write them quickly, which is why Steak ended up being the right tool for that job.

This has all been a really long winded way of saying doing some research before choosing your testing frameworks, or any framework for that matter. Play with it, research it, make sure it meets your goals, not somebody else’s. If you choose the right tools then testing doesn’t need to be scary, complicated, frustrating, etc… Testing is a requirement and a must have, so why not make it fun?

Five years ago I left the world of contracting and reentered the world of the full time employee, and I enjoyed every minute of it (well, almost). Now fast forward five years and I find myself once again at a crossroads. Do I continue on as an FTE or do I become a contractor, and play the field, so to speak? Looks like I’m going to go with the hired gun route for a little while, but that’s not really the point of this post.

During the past week or so I’ve spoken with many great companies and people. I’ve been fortunate enough to have a high degree of interest in what I can bring to the table. During those discussions I talked with a really nice guy at a what seems to be a really cool company, I won’t name names, because this isn’t about either the person or the company, but rather something the engineer said during our phone conversation that got me to thinking.

“We don’t have any tests because I couldn’t convince the company to allocate the time for them.” That statement really hung with me. After I got off the phone I started thinking really hard about that statement, and all I could think of was how testing is not an option and people shouldn’t need to be convinced to have time allocated to them.

As developers it is our responsibility to insist on testing. Always include testing in your time estimates. Never give the client (or your company) an option that includes a time estimate without testing. If a feature takes 2 days to code and a day to write tests, then your estimate is 3 days, never 2. You should never say, “Well, I can get it done in two days if I don’t write any tests.” That’s just an unacceptable thing to say. What you should be saying is, “That feature will take three days to code”.

I don’t feel I should sit here and tell you all the reasons why you should test, you should know them already, and frankly, they’re all very obvious! But, if you need a few bullet points to ‘convince’ your client, here are a few:

• Less bugs – The more tests you have the less bugs you will have. It’s just a fact. You won’t have 100% bug free code, that’s a nearly impossible goal, but you highly reduce the likely hood that as soon as you get your code into production your users will find all the breaking points of your code.
• Better maintainability, means faster feature turn around – When you have a large test suite it means adding, updating, or even removing features because a whole lot easier, which means it SAVES time! Why? Simple, you don’t have to go through and manually test every aspect of your code to make sure you didn’t break something elsewhere by adding that validation, or by refactoring that bit of code, etc… That translates into real $ savings.
• Test driven development saves time – this isn’t quite the same as my last bullet point. Imagine, if you will, you are writing a four step wizard in your application. If you write a few test scripts using something like Cucumber first before you write your code you can simply keep re-running those to make sure your code is working. If you don’t have those test scripts written then you continually have to keep going to a browser and entering all the information in each of the steps so you can test something in step four. Which one do you think takes longer, having a few test scripts you can run, or manually going through the four page wizard each time you make a change?
• It’s an investment – thinking of having test scripts like owning a house. When you don’t have tests and you just keep testing in the browser or the command line what you are doing is a kin to “renting”. There is money being spent, but at the end of the day you have nothing to show for it. You’ve spent hours “testing”, but tomorrow when you come in you have to do it all over again. When you spend those hours writing tests you are actually “buying” something. You have something to show for that time and money you’ve spent. Tomorrow, next week, next month, next year, those scripts will still be there, they’ll still be working for you, giving  you a return on your investment.

Well, I hope I have hopefully made a case to you the engineer as to why you should insist on testing. It’s the right thing to do, for you, for your application and for you client. If if anyone tries to give you grief about it, send them my way, I’ll sort em out!

So with my book, Distributed Programming with Ruby, finally finished, and a nice long weekend I was able to sit down and work on a little pet project of mine. I decided to work on a little site that I could use to track my rather large Pez collection. (Yes, I know, I collect Pez – so what!)

While working on it I got to use some new technologies that I really haven’t had a chance to play, so I thought I would talk a bit about some of the ones I liked the most.

Authlogic

Love it! Finally a decent authentication system! The thing I love most about it? It doesn’t generate a lot of crap in your project. If I were to say one bad thing about it, it would be that it doesn’t generate enough in your project. I know that sounds silly, but it’s the truth. It gives you so much power, without having to generate a ton of lib code and crazy controller code, which is awesome. However, it would be nice if it had a generator that generated a ‘basic’ application for you. Just a little thing, apart from that, love it!

http://github.com/binarylogic/authlogic/tree/master

Cucumber/Webrat

I’m sure by now everyone has heard of Cucumber. I’m not going to pretend that I’m the first to that party! Over the last month or so I’ve really started to use it and it has completely changed my life. That’s not an overstatement.

Cucumber lets you write features and scenario in human readable format. Combine that with Webrat, which lets you do things like click buttons and follow links, you can write some amazing tests that look like something a project manager would write! Brilliant!

These tests beat the hell out of Rails integration tests. Trust me! I love watching Cucumber and Webrat click around my site while I just watch.

http://cukes.info/
http://github.com/brynary/webrat/tree/master

Web App Theme Generator

This cool little plugin helps you to quickly generate a very useful, and laid out, theme for your application. The themes would be familiar to anyone who has used sites like Lighthouse. They’re basic, but they are very well coded and get you on your quickly so you can have something that looks fairly decent.

My only gripe with this plugin is that it is a bit clumsy to use, but thankfully you don’t have to run it very often, only when you create a new controller/resource.

http://gravityblast.com/2009/07/30/2-minutes-admin-layout-with-rails-and-the-web-app-theme-generator/

Delayed Job

The last piece of tech is Delayed Job. I first discovered Delayed Job when I wrote about it in my book. It is a great way to handle and process background tasks. It’s easy, reliable, and scales really well. I’ve been using the Collective Idea fork of the project. It has a generator to create the migration you need. It also has a nice binary to run in the background on your server.

I’ve also been using a little gem I wrote that gives me hooks into Hoptoad, the is_paranoid gem, and a nice subclass for writing workers.

I have been completely enamored with Delayed Job from the first moment I used it, and I’m sure if you haven’t checked it out yet, and you do, you’ll feel the same!

http://github.com/collectiveidea/delayed_job/tree/master
http://github.com/markbates/delayed_job_extras/tree/master

There you go, that’s just a few things I’ve been playing with lately, that I think are going to become mainstays in any Rails project I work on. Hopefully this has given you a little for for thought on things you can use in your next project.