CoverMe – Code Coverage for Ruby 1.9
Ruby 1.9(.2) is an amazing language to develop applications in. It’s faster, more powerful, cleaner, and a huge improvement over Ruby 1.8.x. Because of those reasons every Ruby developer should move to this exciting new version of our language.
Enter CoverMe.
History
While working on a Ruby 1.9/Rails 3 project, and loving everything about it (except for the lack of RCov), I came across a post by Aaron Patterson (of Nokogiri fame). In this post he quickly outlined a very basic coverage tool using the new built-in Coverage module in Ruby 1.9.
After spending a morning playing with it, I was quickly able to grow the idea into something useful for the project. Later that day the company I was consulting for (BiddingForGood.com), and in particular their chief architect, Stuart Garner, told me to take a day or two and clean it up and release it for the world to use, and so here it is.
Features
Here is a brief overview of the features of CoverMe:
Index Page
- Sortable column headers (File, Lines, Lines of Code, Tested %).
- Searching/filtering by file name.
- Filtering by coverage percent.
- Color coded list of files to quickly see which ones are 100% covered, > 90% covered, or less than 90% covered.
- Large color coded average coverage percent, for quick reference.
Detail Page
- Line by line coverage report
- Color coded lines to quickly see which lines where executed and which ones were not.
- Side by side viewing with the corresponding test/spec file (if one exists).
See the README file for more information on installation and usage.
Thanks
I would just quickly like to give another quick thanks to Aaron Patterson for pointing out the Coverage module in Ruby 1.9 and inspiring this, hopefully, helpful little gem. Also another big thanks to Stuart Garner for pushing me to package this up and release it to the world.
Screenshots
Tags: gem, rails, rcov, release, rspec, ruby, testing, tests
August 15th, 2010 at 12:33 pm
Looks good Mark.
Does it play nice with rspec and it’s peers?
Though you may want to update your screenshots once the sentence “This file was is tested nearly enough!” is changed to something more meaningful
August 15th, 2010 at 12:36 pm
It plays very nicely with Rspec, that’s what I developed it with. It should work with Test Unit as well. I’m open to patches if there are issues with other test frameworks.
I also noticed the spelling mistake as well. I fix it in the code, but not in the screenshots. Keen eye.
August 16th, 2010 at 9:11 pm
Mark, this is really interesting. Do you know if there’s any way to see which specific branches are executed on a given line, so that C1 coverage could be measured?
The reason I ask is that alot of times, a line of code is executed, but only one possible branch is followed it still shows as being fully covered. On projects where I absolutely needed to verify the code is solid (like in a foundational library), I’ve been using rcov to verify my coverage is 100%, then using heckle to mutate the code and find branches/conditions that I missed writing specs for. It’s surprising how much you can miss even when being extra careful when writing specs.
While I really like heckle and will still be using it even if I had a way to verify C1 coverage, it would be nice to be able to visually inspect code to see where I haven’t focused enough.
August 17th, 2010 at 8:10 am
Hey Dan, the Coverage module in 1.9 gives you a Hash that looks something like this:
{‘/foo/bar.rb’ => [1, 0, nil, 24, nil, 1]}
The key is the file and the value is an array of integers/nils. Each index of the array represents the corresponding (+1) line in the file. So index 0 is line 1 of the file, index 1 is line 2, and so on… A value of nil means the line is non-executable code, think comments, blank lines, and ‘end’ statements. A value of 0 means the line should have been executed, but wasn’t. A value of 1 or above means the line was executed that number of times. So in my about example line 4 of /foo/bar.rb was executed 24 times.
Does that answer your question? I’m thinking about placing these numbers in the gutter of the report so you can easily see how much each line get executed.
August 20th, 2010 at 6:14 am
Hi Mark
I’m very happy I found your CoverMe gem. However, I have some problems to integrate it into my environment.
My project setup looks like this:
- rvm
- ruby 1.9.2dev (2010-07-11 revision 28618) [x86_64-darwin10.4.0]
- Rails 3.0.0.rc
- Datamapper instead of ActiveRecord
- rspec 2.0.0.beta.19
and a lot of other stuff.
When I add require ‘cover_me’ as the first line in my spec helper file, I get the following output:
Gernot-Koglers-MacBook-Pro:rem2 gernot$ RAILS_ENV=test rake spec
(in /Users/gernot/entwicklung/rem2)
[datamapper] Created database ‘/Users/gernot/entwicklung/rem2/db/test.sqlite3′
[datamapper] Finished auto_migrate! for :default repository ‘/Users/gernot/entwicklung/rem2/db/test.sqlite3′
……
Finished in 0.14481 seconds
6 examples, 0 failures
/Users/gernot/.rvm/gems/ruby-1.9.2-rc2@rem2/gems/cover_me-1.0.0.pre3/lib/cover_me/processor.rb:21:in `block in process!’: undefined method `pattern’ for # (NoMethodError)
from /Users/gernot/.rvm/gems/ruby-1.9.2-rc2@rem2/gems/cover_me-1.0.0.pre3/lib/cover_me/processor.rb:20:in `each’
from /Users/gernot/.rvm/gems/ruby-1.9.2-rc2@rem2/gems/cover_me-1.0.0.pre3/lib/cover_me/processor.rb:20:in `map’
from /Users/gernot/.rvm/gems/ruby-1.9.2-rc2@rem2/gems/cover_me-1.0.0.pre3/lib/cover_me/processor.rb:20:in `process!’
from /Users/gernot/.rvm/gems/ruby-1.9.2-rc2@rem2/gems/cover_me-1.0.0.pre3/lib/cover_me.rb:28:in `block in ‘
rake aborted!
bundle exec /Users/gernot/.rvm/rubies/ruby-1.9.2-rc2/bin/ruby -Ilib -Ispec “./spec/models/adresse_spec.rb” “./spec/models/person_spec.rb” failed
When I put the require after require ‘rspec/rails’, I get no errors but the coverage report is empty.
When I put the require into environments/test.rb, I get
Gernot-Koglers-MacBook-Pro:rem2 gernot$ RAILS_ENV=test rake spec
(in /Users/gernot/entwicklung/rem2)
rake aborted!
key not found: :adapter
(See full trace by running task with –trace)
Gernot-Koglers-MacBook-Pro:rem2 gernot$
Datamapper does not setup the test database and no specs are executed, but I get a perfect coverage report.
Could there be a dependency between CoverMe and ActiveRecord?
August 20th, 2010 at 6:38 am
Will it work with Cucumber features? This is critical for me as I tend to rely on cucumber features for my code test coverage (lots of my code isn’t touched by my limited rspec tests, but I try to have 100% covered by my .feature files).
I haven’t moved to Ruby 1.9 yet, but having a solid coverage tool like this might be the prompting I need to make the move. Thanks.
August 23rd, 2010 at 11:46 am
I installed this and found it to be great…until I changed my code and re-ran my tests. The code coverage reports didn’t change to reflect my new code/tests. I deleted the coverage folder entirely and re-ran tests, and it didn’t re-create the folder with new reports.
I’ve got the gem installed (v 1.0.0.pre3) and have “require ‘cover_me’” in my spec_helper.rb. I’m using “rspec spec/” to run tests…am I missing something else?
Thanks!
August 23rd, 2010 at 12:52 pm
@Gernot Kogler and @SH can you both please open tickets on your issues so I can resolve them?
http://github.com/markbates/cover_me/issues
Please make sure to include as much detail about your environments as possible. Also, please make sure that when you’re requiring ‘cover_me’ that is the very FIRST line of your spec_helper.rb or test_helper.rb file. That’s EXTREMELY important.
Thanks.
August 23rd, 2010 at 12:54 pm
@Grant Neufeld it should work with cucumber you just have to make sure that ‘Cover Me’ is required before anything else. The Ruby 1.9 coverage library only gather coverages on files that are have been required after it has been loaded, so making sure you require Cover Me first is extremely important. After that, it shouldn’t matter if you’re using cucumber, or any other library for that matter. Hope that helps.