Meta Bates

“Testing is painful.”

“Testing is hard.”

“Testing is complicated.”

“Testing is not fun.”

I hear those sorts of things all the time when I talk to people about testing. I agree that sometimes testing can be all of those things, but if you choose the right tools, the tools that best suite you, testing doesn’t have to be. Let me give you an example of what I’m talking about, how choosing the right tools can make a huge impact on how you feel about testing.

When working for a client recently I came across the need for end to end integration testing. I needed to test, amongst other things, the flow of a user registering through the application in a few different ways. Because registration behaves differently based on where you come from and where you want to go, I needed a good way to test that entire flow, so simple unit and functional tests just were not going to cut it.

In the Ruby community there is a big push to use a testing framework called, Cucumber. Cucumber is a behavior driven development (BDD) tool that let’s you write user stories in plain English. Those stories then get translated into Ruby code that tests those stories against your application. Because of it’s popularity, and some of it’s quite amazing features, this was my first stop on the path to integration testing bliss.

Let me give you an example of a Cucumber script:

Feature: Registration
  In order to use My Great Application
  As a user
  I want to be able to register
  
  Scenario: 'Standard Registration'
    Given I am not currently logged in
    When I am on the signup page
    Then I should see "Sign Up"
    And I fill in "Name (required)" with "Mickey Dolenz"
    And I fill in "Email (required)" with "mickey@monkees.com"
    And I fill in "Password (required)" with "password"
    And I fill in "Password confirmation" with "password"
    And I press "Register"
    Then I should see "Sign Up - Confirm Your Account"
    Then I should be on the registration thank you page
    Then "mickey@monkees.com" should receive an email
    When I open the email
    Then I should see "Confirm my account" in the email body
    When I follow "Confirm my account" in the email
    Then I should be on the welcome page
    And I should see "Welcome to the Great Application"
    
  Scenario: 'Accepting an invitation'
    Given I am not currently logged in
    And the "Boys and Girls Club" invites "mickey@monkees.com" to join
    Then "mickey@monkees.com" should receive an email
    When I open the email
    Then I should see "Accept Invitation" in the email body
    When I follow "Accept Invitation" in the email
    Then I should be on the signup page
    Then I should see "Sign Up"
    And I fill in "Name (required)" with "Mickey Dolenz"
    And the "Email (required)" field should contain "mickey@monkees.com"
    And I fill in "Password (required)" with "password"
    And I fill in "Password confirmation" with "password"
    And I press "Register"
    Then the account "mickey@monkees.com" should be "activated"
    Then I should be on the accept/decline invitation page
    And I should see "Join the Boys and Girls Club"

That script tests the user registration flow through an application in a couple of different ways, first through ‘standard’ registration, and then through being invited to join. Now, the beauty of Cucumber is that these scripts are ‘human’ readable. Your product manager, or other stake holders, should be able to write these scripts themselves, and you, the developer, should be able to just plug them in and code until those scripts pass.

Unfortunately, while that sounds like a little slice of Heaven, the reality is far from it in practice. First, getting stake holders to actually write these ‘stories’, as their typically called, is a tough chore to begin with. If they do write them, they’re typically not going to be ‘plug and play’. Why? Well, when Cucumber reads these scripts it goes line by line and tries to find some code that matches the regular expression of that line and then execute it. If it doesn’t find matching code, then it fails. That means that your stake holders need to write these scripts in a very particular way or developers need to sit down and massage those stories to fit the correct regular expression.

Now, let me just take this opportunity to say that this is not a post about how much I hate Cucumber, in fact I think Cucumber is a pretty amazing piece of software, and does in fact have a lot of great uses. Instead, what I’m talking about it is how Cucumber turned out not to be the right tool for the job for me on a recent project.

So why wasn’t Cucumber the right tool for the job? Great question, glad you asked. Cucumber turned out not to be the right tool for a few reasons. The biggest of which was that I was the one who was writing the user stories. The stake holders had no desire to write these stories, which meant I had to write them. The I had to write the ‘steps’ that back each line of the script. In all fairness, Cucumber does give you some great steps right out of the box. After some fiddling I finally got the Cucumber scripts up and running and testing my work flow. But I definitely ran into some issues.

Because Cucumber isn’t pure Ruby I had a hard time doing something as simple as just printing out the request’s body and headers without having to write a step that did just that, then add that step to my story, etc… It’s overall fiddlyness and non-intuitive way of doing things caused me a lot of grief and time. And, most importantly, I wasn’t really getting the big benefit of Cucumber, stake holder’s writing the stories. So I was doing all this work and not getting the benefits of Cucumber.

So what did I do? I turned to a library called Steak. Steak allows you to write integration tests using pure Ruby and integrates directly in with RSpec, my preferred testing framework. With Steak I was able to write my integration tests in just a few minutes.

require File.expand_path(File.dirname(__FILE__) + '/acceptance_helper')

feature "Registration Steak", %q{
  In order to use My Great Application
  As a user
  I want to be able to register
} do

  scenario "register throught the standard registration process" do
    visit signup_path
    
    expect {
      fill_in_registration(:name => 'Mickey Dolenz', :email => 'mickey@monkees.com')
    }.to change(delivered_emails, :size).by(1)
    
    user = User.find_by_email('mickey@monkees.com')
    user.should be_unverified
    
    page.should have_content('You have signed up successfully. However, we could not sign you in because your account is unconfirmed.')
    current_path.should == root_path
    
    visit signin_path
    fill_in 'user_email', :with => 'mickey@monkees.com'
    fill_in 'user_password', :with => 'password'
    click_button 'Sign In'
    
    page.should have_content('You have to confirm your account before continuing.')
    current_path.should == new_user_session_path

    click_email_link_matching(/users\/confirmation/, delivered_emails.first)

    current_path.should == welcome_users_path
    page.should have_content('Welcome to the Great Application')
    user.reload
    user.should be_activated
  end
  
  scenario "register from an invitation" do
    mark = users(:mark)
    organization = mark.organizations.first
    signin(mark)
    
    visit new_organization_invitation_path(organization)
    expect {
      expect {
        fill_in 'organization_invitation_worker_emails', :with => 'mickey@monkees.com'
        click_button 'Send Invites'
        DJ.first.invoke_job
      }.to change(delivered_emails, :size).by(1)
    }.to change(organization.invitations, :count).by(1)
    
    invitation = organization.invitations.first
    
    signout
    
    click_first_link_in_email(delivered_emails.first)
    
    current_path.should == signup_path

    fill_in_registration(:name => 'Mickey Dolenz')
    
    current_path.should == organization_invitation_path(organization, invitation)
    
    user = User.find_by_email('mickey@monkees.com')
    user.should be_activated
    
    expect {
      click_on 'Get Started Now!'
    }.to change(user.organizations, :count).by(1)
    
    current_path.should == organization_campaign_path(organization, organization.campaigns.first)
  end
  
end

While my Steak scripts a bit more wordy and are definitely not ‘human’ readable and editable by stakeholders, they did achieve my goal of allowing me to write integration tests quickly.

So here you see I picked a very powerful tool, that has a lot of great benefits, Cucumber, but I picked it for the wrong reasons. I picked it because it was popular, and not because it would help me achieve my goals. If my goals where to have stakeholders write the stories and hand them off to development, than it would’ve been a better choice. But in the end my goal was to write integration tests and write them quickly, which is why Steak ended up being the right tool for that job.

This has all been a really long winded way of saying doing some research before choosing your testing frameworks, or any framework for that matter. Play with it, research it, make sure it meets your goals, not somebody else’s. If you choose the right tools then testing doesn’t need to be scary, complicated, frustrating, etc… Testing is a requirement and a must have, so why not make it fun?

So back in the dark ages of my career, pre-2006, I spent a long time coding Java. Yeah, I know, please don’t judge. Anyway, In Java, for those of you who are unaware were two constructs that I occasionally wish I had in Ruby, those are Interfaces and Abstract Classes. The difference between these two constructs is subtle, but important.

In Java an Interface is a basically a blueprint of methods that the class who implements the Interface needs to implement. For example:

interface Bicycle {

  void changeGear(int newValue);

  void speedUp(int increment);

  void applyBrakes(int decrement);
}

public class ACMEBicycle implements Bicycle {
   
   public void changeGear(int newValue) {
     // do some work here
   }
   
   public void speedUp(int increment) {
     // do some work here
   }
   
   public void applyBrakes(int decrement) {
     // do some work here
   }

}

Here we have a Bicycle Interface that says there are three methods that need to be implemented. It is then the responsibility of the ACMEBicycle class to implement those methods. Now, an Abstract Class in Java is similar to an Interface in that it too is a blueprint of methods that the extending class may or may not need to implement. There in lies one of the differences between the two. Let’s take a look at the same example, but this time we want to implement the same behavior of all of our extending classes for the applyBrakes method:

abstract class Bicycle {
  
  abstract public void changeGear(int newValue);

  abstract public void speedUp(int increment);

  public void applyBrakes(int decrement) {
    // do some work here
  }
  
}

public class ACMEBicycle extends Bicycle {
   
   public void applyBrakes(int decrement) {
     // do some work here
   }

}

An Abstract Class is a great way to provide a mix of fully implemented methods as well as providing subclasses with a mixture of methods that need to be implemented by the extending class.

The really powerful part of all of this is two fold. First, the Java compiler will happily yell at you and fail if it finds that you haven’t implemented some of the methods that you were told you had to. Second, you can easily see the methods that you need to document right there, you can even copy/paste their definitions right into your class so you can start to fill them out.

So, how does this bring us over to Ruby? Great question. I’d like to take a few moments and explore a few ways we can get some of this power in Ruby.

Unfortunately, or fortunately depending on how you look at it (I see it as a mixed blessing), there is no compiler in Ruby, so we don’t really have a good way of having the system yell at us if we don’t implement the methods we were supposed to. But, there is still plenty we can do to help those who are implementing our classes both know what they need to implement and to find out what they haven’t implemented when their program is executing.

Here is one implementation on we can gain a bit of that functionality back in Ruby:

module AbstractInterface
  
  class InterfaceNotImplementedError < NoMethodError
  end
  
  def self.included(klass)
    klass.send(:include, AbstractInterface::Methods)
    klass.send(:extend, AbstractInterface::Methods)
  end
  
  module Methods
    
    def api_not_implemented(klass)
      caller.first.match(/in \`(.+)\'/)
      method_name = $1
      raise AbstractInterface::InterfaceNotImplementedError.new("#{klass.class.name} needs to implement '#{method_name}' for interface #{self.name}!")
    end
    
  end
  
end

class Bicycle
  include AbstractInterface
  
  # Some documentation on the change_gear method
  def change_gear(new_value)
    Bicycle.api_not_implemented(self)
  end
  
  # Some documentation on the speed_up method
  def speed_up(increment)
    Bicycle.api_not_implemented(self)
  end
  
  # Some documentation on the apply_brakes method
  def apply_brakes(decrement)
    # do some work here
  end
  
end

class AcmeBicycle < Bicycle
end

bike = AcmeBicycle.new
bike.change_gear(1) # AbstractInterface::InterfaceNotImplementedError: AcmeBicycle needs to implement 'change_gear' for interface Bicycle!

What we’ve done here is to inject a Module into our Bicycle class to give it a nice error it can raise and a little bit of help building a nice error message for the user. Then in our Bicycle class we define all the methods we want and in the ones we need the end user to define we can call the api_not_implemented method and it will raise the AbstractInterface::InterfaceNotImplementedError error for us.

We could simplify this a bit by having a nice little helper macro that we can use to build these methods, like this:

module AbstractInterface
  
  class InterfaceNotImplementedError < NoMethodError
  end
  
  def self.included(klass)
    klass.send(:include, AbstractInterface::Methods)
    klass.send(:extend, AbstractInterface::Methods)
    klass.send(:extend, AbstractInterface::ClassMethods)
  end
  
  module Methods
    
    def api_not_implemented(klass, method_name = nil)
      if method_name.nil?
        caller.first.match(/in \`(.+)\'/)
        method_name = $1
      end
      raise AbstractInterface::InterfaceNotImplementedError.new("#{klass.class.name} needs to implement '#{method_name}' for interface #{self.name}!")
    end
    
  end
  
  module ClassMethods
    
    def needs_implementation(name, *args)
      self.class_eval do
        define_method(name) do |*args|
          Bicycle.api_not_implemented(self, name)
        end
      end
    end
    
  end
  
end

class Bicycle
  include AbstractInterface
  
  needs_implementation :change_gear, :new_value
  needs_implementation :speed_up, :increment
  
  # Some documentation on the apply_brakes method
  def apply_brakes(decrement)
    # do some work here
  end
  
end

class AcmeBicycle < Bicycle
end

bike = AcmeBicycle.new
bike.change_gear(1) # AbstractInterface::InterfaceNotImplementedError: AcmeBicycle needs to implement 'change_gear' for interface Bicycle!

That approach certainly makes our code look a bit cleaner, I’m not denying that, however it has one really big flaw, at least for me anyway, it doesn’t give us a good to place to hang our documentation hat. In the previous approach we had actual methods that we could then document and that documentation would then show up in RDoc when it’s outputted. With the latter approach, however, we can document the hell out of the needs_implementation calls we have in the Bicycle class, but they won’t ever show up in the documentation. That means that users of our library have to crack open the actual code itself to see what it they are expected to implement.

Another approach we could’ve taken, which I bother to demonstrate here as I don’t think it offers a better approach is to have the needs_implementation method collect up the names of those methods and use method_missing to report that the method needs to be implemented. I mention it here only for completeness, but it definitely is not the best solution to this problem.

Finally, I would like to note that, as far as I can see, there is no way in Ruby to create a callback hook for when a class has been defined. If there was in fact such a hook we could use to it immediately notify the end user that they have forgotten to implement certain methods. Perhaps in Ruby 2.0??? That’s just pure hope on my part.

That’s it. I hope you enjoyed our brief (*cough*) look through implementing Interface and Abstract Classes in Ruby. I hope you’ve enjoyed it.

* PS, yes, I’m aware I didn’t talk about multiple vs. single inheritance in either Java or Ruby, nor did I talk about the fact that in Ruby you can’t really have Abstract Classes. I thought that was all a bit much for an already rather lengthy post as it was. Perhaps another day.

Last week I received an email from someone who used to work at a company that I used to work with. I didn’t know him, but he knew me through my work at the company, and my other exploits. He sent me an email to say that after a short time with the company he had been laid off, along with half of the development team. He wasn’t looking for pity, but rather advice.

What kind of advice was he asking for, well, he quite simply needed to know how could he become an ‘expert’ Ruby on Rails developer. First, let me say that this post won’t have anything to do with Ruby, Rails, or any other specific programming language. Everything I’ll talk about is valid in ANY language on ANY platform. With that disclaimer, let’s move on, shall we?

While at this company he got introduced to Ruby on Rails and really loved it, coming from a non-Rails background. Since being laid off he’s been trying to land another Rails gig, but everyone is looking for ‘expert’ Rails developers. So the question was, how to become an ‘expert’ developer?

I keep putting ‘expert’ in quotes because, let’s be honest here, that’s a VERY subjective term. As someone who has hired many developers in his day, I can tell you that I’v

e hired newbies to ‘experts’ and everywhere in between. Everyone has their merits and possibilities. I’ve met ‘experts’ that I wouldn’t hire to take out my trash, let alone build my business. I’ve also met people right out of college that I would hire again and again. So your mileage my vary.

So… how do you build up that ‘expert’ reputation? Let’s look at it. Below are some of things I’ve done, as well as some of the things that I look for as a hiring manager. Some are incredibly easy to do, others require work, but in the end they WILL pay off, and you’ll easily be at the head of the pack when going for that job.

Build Something

When you are looking for a job people want to see what it is you’ve actually built. If you haven’t built anything, then how are you an expert? Build a lot of different things and put them up on the web for perspective employers to find and play around with. Use these are a playground for trying out all those cool new technologies you keep hearing about. Want to give NoSQL a try? Great, build an app that uses it. Need to improve your testing chops? Write an application and write all the tests you can possibly think of!

Get a GitHub Account

I can’t tell you how important GitHub has become when trying to make a name for yourself. It seems like unless you’re on GitHub, you’re nobody. While that might not be true, it certainly hurts more than it helps to not have an account. You know those apps you’ve just been building and playing around with? Post them on GitHub! Then put your profile page link on your resume. Yep, you read that right. Give those looking at your resume a link to your code. Let them see how good a developer you actually are. Show them you know how to code all the things you’ve got on your resume. Listing a language, platform, or tool on resume is one thing, but actually showing your perspective employer is another! They’ll love it.

While you’re on GitHub, why not contribute to an open source project that’s up there. There are plenty of them, and they’re ALL looking for people to help out with their projects. Simply fork the projects, make some improvements, and then give those changes back to the projects owner. This looks great on a resume and really helps to show that you are interested and active in the community. Again, employers love this! Plus, you’ll be starting to build a name for yourself, and building a network, and a network is INCREDIBLY important when looking for work.

Blog and Write

I should probably heed my own advice here and blog more often, but do as I say, not as I do. With that said I wrote a book, which looks AMAZING on a resume, but might be a bit out of reach for most people, so I recommend blogging instead. Why should you blog? Well, it shows that you have good communications skills, again very important to most employers. It can also show that you have a deep understanding of whatever it is you blog about.

What should you write about? If you’re stuck on a topic, might I make a recommendation or two. First, when you’re building those applications I mentioned early if you run into a bug or something else that got you a bit stuck, blog about it! Others could really benefit from your experience. Explain the problem and how you went about solving it. Another great thing to write about is your favorite libraries or plugins. Pick a different one each week and dissect it. Write about how it works, what it does, etc… This is a great exercise in both writing and learning about how things work. Very valuable.

Network

I mentioned earlier that a good network is INCREDIBLY important when looking for work, and I wasn’t lying. It’s the most important thing. A good network will constantly be feeding you new opportunities, or putting you in touch with those who can. So how do you develop that network? A few ways, I mentioned contributing to open source earlier, that’s a great way. Another great way is through conferences, hackfests, rumbles, and whatever other local (and non-local) events are being held in your development community of choice. Attend these events, participate, introduce yourself, speak, buy drinks, whatever! Just get out there and NETWORK!!

Conclusion

The gentleman who emailed me said that he was reading a lot of books in hopes of becoming an ‘expert’. While I’m not going to tell you not to read books (you should!!), I will tell you that there is no substitute for doing. All of things I’ve talked about above are ALL about doing. Reading is not doing, it’s reading. It’s passive and will not get you further in your career. There’s no place on a resume for the books you’ve read. Take what you’ve read and put it into action, then you’ll be on your way to becoming an ‘expert’ developer.

In a previous post, Testing Is Not An Option, I talked a lot about why you should write tests, and the arguments you can put forth to your client, manager, or whoever it may be as to why you should write tests. What I didn’t talk about was how to start writing tests. So let’s talk about that for a bit, shall we?

When I’m talking with a potential client, well at least a client that has an existing code base, I always ask what their code coverage stats are. Now, I know at code coverage stats aren’t the be all end all of measuring how good your tests are, but they’re a basic enough metric to use as a guide. If they say they’re high, then I usually dig in more about how they’re testing; what frameworks, BDD, TDD, that sort of thing. Usually though I get a few minutes where they apologize and sheepishly give me their reasons for having little or no tests.

Here are few of those reasons:

• We don’t/didn’t have the time.
• We don’t know how.
• It was/is too complicated.
• It was/is too overwhelming.

Let’s talk about each of this points for a minute.

“We don’t/didn’t have the time.”

I never accept time as an argument against testing. Testing ends up repaying it’s time investment, and will ultimately give you more time than if you didn’t write code. It’s a win-win. Again see my previous post in how to get the time signed off on as part of t he project timeline.

“We don’t know how.”

Learn. There’s no better time than the present and no better way to learn than being thrown into the deep end. The web is crawling with documentation, screen casts, how to articles and tutorials, and there are plenty of books to get you going. In short the k knowledge is literally at your finger tips, and to be honest it’s easier than you think.

“It was/is too complicated.”

That usually means you’re doing it wrong. Take a step back and assess what it is you’re trying to do. You’re tests should be simple and concise. Don’t write tests that are hundreds of lines long. They’re tests, not entrance exams to MIT.

“It was/is too overwhelming.”

Certainly if you didn’t write tests as you went along it can get quite overwhelming thinking about all the tests you now need to write for your monolithic app. I’ll talk about how you can solve that problem in a minute.

Making It Happen

Ok, so now that we’ve identified a few of the excuses let’s talk about how you can starting writing tests today for your application. So, take a deep breath and let’s begin.

If you’re staring at an existing application, don’t try to tackle it all at once, you’ll just get overwhelmed, scared, and confused. Instead take it one file/class at a time. First start with your models, as this is where the majority of your application business logic should be. Alphabetically each day pick the next class (or a couple of them) in the list and start to fill our your test files.

What do I mean by fill out your tests files, I mean creating pending tests for each of the methods of your model. Here’s an example of a basic Ruby* class and what the pending RSpec spec file would look like:

# Class: class Entity def tax_id if self.person? # code here else # code here end end def person? # code here end end # Spec: describe Entity do describe "tax_id" do it "should return a Social Security number if the entity is a Person" do pending end it "should return a Tax ID number if the entity is a Corporation" do pending end end describe "person?" do it "should return true if the entity is a person" do pending end it "should return false if the entity is not a person" do pending end end end

Notice how the method that has the if/else statement in it has two pending tests for it. We need to test each variation of the method.

Now when you run your tests you’ll see that you have a bunch of pending tests. Great! Now you just need to fill them in, but at least you know what should be filled in.

I also recommend that you do this every time you create a new method. As soon as you give your method a name go to your corresponding test and create a pending test for that method. This way you know that you have to test that method later. (In a perfect world I would love to see you write your test before returning to your class to fill in the method itself, but baby steps for now.)

Once you have all your pending tests setup each day try to fill in the details of each pending test for a whole class. If that’s too much, then try to set aside an hour a day and fill in as many pending tests as you can. Alternatively you can also fill in the tests during the course of the day as you use one of the methods without tests.

Another great way to start filling in your test suite is each time you get a new bug, write a test to reproduce it. This is a great habit to get into as you’ll eventually have a great suite of regression tests in place to help prevent those nasty bugs from returning. Write the test, see that it fails, then fix your bug. When your teat passes then you know you’ve fixed the bug!

Finally, start small. Start by writing unit tests. These are the types of tests I just described. They test a very particular part of your code base to ensure that it does what it should do. These tests are typically easy to write and act as a great corner stone to your test suite as a whole. Don’t try to jump right in with full integration tests. The frameworks typically have a steep learning curve, and are more complicated to get up and running. This will lead to frustration and the old feeling of being overwhelmed. You can add these tests in later as you gain experience.

Well, there you have it, a few simple tricks to help you get started testing today. I know this post was a bit on the lengthy side, but I’m glad you stuck with me. Your life will be better for it. When you have a large and expansive test suite life is just better. Food tastes better. The sky is bluer. There will be a skip in your step. And you can use your incredibly high code coverage stats as a pick up line in a bar. On second thought, scratch that last thought. I wrote a test to see if that would work and it failed. It failed miserably.

* Please not that while I use Ruby as the example language here, the concept applies to whatever language you use.

In August I announced CoverMe a code coverage tool for Ruby 1.9. Well, today I announce that it has hit it’s first release candidate! I’ve very excited by the fact it’s getting close to an ‘official’ release.

The response to CoverMe has been great and through feedback from the community I’ve made a lot of improvements and fixed a lot of issues.

While quite a few things have changed under the hood, not much has changed in how you use CoverMe.

Installation

The following are instructions for how you would configure CoverMe for a Rails 3 project, adjust to your local environment accordingly.

In  your Gemfile add the following:

gem 'cover_me', '>= 1.0.0.rc1', :group => :test

Then run:

$ bundle install

After CoverMe is installed place the following line at the VERY TOP of your ‘test_helper.rb’ or ‘spec_helper.rb’ file (for Cucumber put it at the top of the ‘env.rb’ file):

require 'cover_me'

I can’t emphasize enough how important it is that the require statement is at the VERY top of that file!

Finally (and optionally) run:

$ rails g cover_me:install

This will simply install a Rake task that will wrap both Test::Unit and RSpec tasks with CoverMe and will launch the results at the end of the test suites. I would recommend it. It’s kinda the whole point.

That’s it!

Enjoy the release candidate, and of course, please let me know if you find any issues with it. Issues can be reported on here.