“Testing is painful.”
“Testing is hard.”
“Testing is complicated.”
“Testing is not fun.”
I hear those sorts of things all the time when I talk to people about testing. I agree that sometimes testing can be all of those things, but if you choose the right tools, the tools that best suite you, testing doesn’t have to be. Let me give you an example of what I’m talking about, how choosing the right tools can make a huge impact on how you feel about testing.
When working for a client recently I came across the need for end to end integration testing. I needed to test, amongst other things, the flow of a user registering through the application in a few different ways. Because registration behaves differently based on where you come from and where you want to go, I needed a good way to test that entire flow, so simple unit and functional tests just were not going to cut it.
In the Ruby community there is a big push to use a testing framework called, Cucumber. Cucumber is a behavior driven development (BDD) tool that let’s you write user stories in plain English. Those stories then get translated into Ruby code that tests those stories against your application. Because of it’s popularity, and some of it’s quite amazing features, this was my first stop on the path to integration testing bliss.
Let me give you an example of a Cucumber script:
Feature: Registration
In order to use My Great Application
As a user
I want to be able to register
Scenario: 'Standard Registration'
Given I am not currently logged in
When I am on the signup page
Then I should see "Sign Up"
And I fill in "Name (required)" with "Mickey Dolenz"
And I fill in "Email (required)" with "mickey@monkees.com"
And I fill in "Password (required)" with "password"
And I fill in "Password confirmation" with "password"
And I press "Register"
Then I should see "Sign Up - Confirm Your Account"
Then I should be on the registration thank you page
Then "mickey@monkees.com" should receive an email
When I open the email
Then I should see "Confirm my account" in the email body
When I follow "Confirm my account" in the email
Then I should be on the welcome page
And I should see "Welcome to the Great Application"
Scenario: 'Accepting an invitation'
Given I am not currently logged in
And the "Boys and Girls Club" invites "mickey@monkees.com" to join
Then "mickey@monkees.com" should receive an email
When I open the email
Then I should see "Accept Invitation" in the email body
When I follow "Accept Invitation" in the email
Then I should be on the signup page
Then I should see "Sign Up"
And I fill in "Name (required)" with "Mickey Dolenz"
And the "Email (required)" field should contain "mickey@monkees.com"
And I fill in "Password (required)" with "password"
And I fill in "Password confirmation" with "password"
And I press "Register"
Then the account "mickey@monkees.com" should be "activated"
Then I should be on the accept/decline invitation page
And I should see "Join the Boys and Girls Club"
That script tests the user registration flow through an application in a couple of different ways, first through ‘standard’ registration, and then through being invited to join. Now, the beauty of Cucumber is that these scripts are ‘human’ readable. Your product manager, or other stake holders, should be able to write these scripts themselves, and you, the developer, should be able to just plug them in and code until those scripts pass.
Unfortunately, while that sounds like a little slice of Heaven, the reality is far from it in practice. First, getting stake holders to actually write these ‘stories’, as their typically called, is a tough chore to begin with. If they do write them, they’re typically not going to be ‘plug and play’. Why? Well, when Cucumber reads these scripts it goes line by line and tries to find some code that matches the regular expression of that line and then execute it. If it doesn’t find matching code, then it fails. That means that your stake holders need to write these scripts in a very particular way or developers need to sit down and massage those stories to fit the correct regular expression.
Now, let me just take this opportunity to say that this is not a post about how much I hate Cucumber, in fact I think Cucumber is a pretty amazing piece of software, and does in fact have a lot of great uses. Instead, what I’m talking about it is how Cucumber turned out not to be the right tool for the job for me on a recent project.
So why wasn’t Cucumber the right tool for the job? Great question, glad you asked. Cucumber turned out not to be the right tool for a few reasons. The biggest of which was that I was the one who was writing the user stories. The stake holders had no desire to write these stories, which meant I had to write them. The I had to write the ‘steps’ that back each line of the script. In all fairness, Cucumber does give you some great steps right out of the box. After some fiddling I finally got the Cucumber scripts up and running and testing my work flow. But I definitely ran into some issues.
Because Cucumber isn’t pure Ruby I had a hard time doing something as simple as just printing out the request’s body and headers without having to write a step that did just that, then add that step to my story, etc… It’s overall fiddlyness and non-intuitive way of doing things caused me a lot of grief and time. And, most importantly, I wasn’t really getting the big benefit of Cucumber, stake holder’s writing the stories. So I was doing all this work and not getting the benefits of Cucumber.
So what did I do? I turned to a library called Steak. Steak allows you to write integration tests using pure Ruby and integrates directly in with RSpec, my preferred testing framework. With Steak I was able to write my integration tests in just a few minutes.
require File.expand_path(File.dirname(__FILE__) + '/acceptance_helper')
feature "Registration Steak", %q{
In order to use My Great Application
As a user
I want to be able to register
} do
scenario "register throught the standard registration process" do
visit signup_path
expect {
fill_in_registration(:name => 'Mickey Dolenz', :email => 'mickey@monkees.com')
}.to change(delivered_emails, :size).by(1)
user = User.find_by_email('mickey@monkees.com')
user.should be_unverified
page.should have_content('You have signed up successfully. However, we could not sign you in because your account is unconfirmed.')
current_path.should == root_path
visit signin_path
fill_in 'user_email', :with => 'mickey@monkees.com'
fill_in 'user_password', :with => 'password'
click_button 'Sign In'
page.should have_content('You have to confirm your account before continuing.')
current_path.should == new_user_session_path
click_email_link_matching(/users\/confirmation/, delivered_emails.first)
current_path.should == welcome_users_path
page.should have_content('Welcome to the Great Application')
user.reload
user.should be_activated
end
scenario "register from an invitation" do
mark = users(:mark)
organization = mark.organizations.first
signin(mark)
visit new_organization_invitation_path(organization)
expect {
expect {
fill_in 'organization_invitation_worker_emails', :with => 'mickey@monkees.com'
click_button 'Send Invites'
DJ.first.invoke_job
}.to change(delivered_emails, :size).by(1)
}.to change(organization.invitations, :count).by(1)
invitation = organization.invitations.first
signout
click_first_link_in_email(delivered_emails.first)
current_path.should == signup_path
fill_in_registration(:name => 'Mickey Dolenz')
current_path.should == organization_invitation_path(organization, invitation)
user = User.find_by_email('mickey@monkees.com')
user.should be_activated
expect {
click_on 'Get Started Now!'
}.to change(user.organizations, :count).by(1)
current_path.should == organization_campaign_path(organization, organization.campaigns.first)
end
end
While my Steak scripts a bit more wordy and are definitely not ‘human’ readable and editable by stakeholders, they did achieve my goal of allowing me to write integration tests quickly.
So here you see I picked a very powerful tool, that has a lot of great benefits, Cucumber, but I picked it for the wrong reasons. I picked it because it was popular, and not because it would help me achieve my goals. If my goals where to have stakeholders write the stories and hand them off to development, than it would’ve been a better choice. But in the end my goal was to write integration tests and write them quickly, which is why Steak ended up being the right tool for that job.
This has all been a really long winded way of saying doing some research before choosing your testing frameworks, or any framework for that matter. Play with it, research it, make sure it meets your goals, not somebody else’s. If you choose the right tools then testing doesn’t need to be scary, complicated, frustrating, etc… Testing is a requirement and a must have, so why not make it fun?
Because I maintain several open source projects on Github I’m constantly getting emailed questions or issues, or people are always opening up tickets with bugs, issues, complaints, etc… And I really appreciate the feedback on these projects, I really do. What I would appreciate more is if instead of just opening a ticket, or sending an email, why not fork the project, fix it, and then contact me?
So, how do I feel about CoffeeScript being included with Rails? Well, I’ll get to that. Let’s go over some of the most common arguments I’ve heard from people this week about why they’re anti-CoffeeScript in Rails.
