Fork me on GitHub

Posts Tagged ‘tests’

Testing is NOT an Option

Thursday, July 1st, 2010

Five years ago I left the world of contracting and reentered the world of the full time employee, and I enjoyed every minute of it (well, almost). Now fast forward five years and I find myself once again at a crossroads. Do I continue on as an FTE or do I become a contractor, and play the field, so to speak? Looks like I’m going to go with the hired gun route for a little while, but that’s not really the point of this post.

During the past week or so I’ve spoken with many great companies and people. I’ve been fortunate enough to have a high degree of interest in what I can bring to the table. During those discussions I talked with a really nice guy at a what seems to be a really cool company, I won’t name names, because this isn’t about either the person or the company, but rather something the engineer said during our phone conversation that got me to thinking.

“We don’t have any tests because I couldn’t convince the company to allocate the time for them.” That statement really hung with me. After I got off the phone I started thinking really hard about that statement, and all I could think of was how testing is not an option and people shouldn’t need to be convinced to have time allocated to them.

As developers it is our responsibility to insist on testing. Always include testing in your time estimates. Never give the client (or your company) an option that includes a time estimate without testing. If a feature takes 2 days to code and a day to write tests, then your estimate is 3 days, never 2. You should never say, “Well, I can get it done in two days if I don’t write any tests.” That’s just an unacceptable thing to say. What you should be saying is, “That feature will take three days to code”.

I don’t feel I should sit here and tell you all the reasons why you should test, you should know them already, and frankly, they’re all very obvious! But, if you need a few bullet points to ‘convince’ your client, here are a few:

  • Less bugs – The more tests you have the less bugs you will have. It’s just a fact. You won’t have 100% bug free code, that’s a nearly impossible goal, but you highly reduce the likely hood that as soon as you get your code into production your users will find all the breaking points of your code.
  • Better maintainability, means faster feature turn around – When you have a large test suite it means adding, updating, or even removing features because a whole lot easier, which means it SAVES time! Why? Simple, you don’t have to go through and manually test every aspect of your code to make sure you didn’t break something elsewhere by adding that validation, or by refactoring that bit of code, etc… That translates into real $ savings.
  • Test driven development saves time – this isn’t quite the same as my last bullet point. Imagine, if you will, you are writing a four step wizard in your application. If you write a few test scripts using something like Cucumber first before you write your code you can simply keep re-running those to make sure your code is working. If you don’t have those test scripts written then you continually have to keep going to a browser and entering all the information in each of the steps so you can test something in step four. Which one do you think takes longer, having a few test scripts you can run, or manually going through the four page wizard each time you make a change?
  • It’s an investment – thinking of having test scripts like owning a house. When you don’t have tests and you just keep testing in the browser or the command line what you are doing is a kin to “renting”. There is money being spent, but at the end of the day you have nothing to show for it. You’ve spent hours “testing”, but tomorrow when you come in you have to do it all over again. When you spend those hours writing tests you are actually “buying” something. You have something to show for that time and money you’ve spent. Tomorrow, next week, next month, next year, those scripts will still be there, they’ll still be working for you, giving  you a return on your investment.

Well, I hope I have hopefully made a case to you the engineer as to why you should insist on testing. It’s the right thing to do, for you, for your application and for you client. If if anyone tries to give you grief about it, send them my way, I’ll sort em out!

Temporary fix for the DataMapper bug

Wednesday, April 9th, 2008

If like me you’re plagued by this DataMapper bug that kills your test cases. I’ve packaged up a temporary version of the DataMapper gem that seems to fix the problem.

WARNING: This gem is NOT sanctioned by the DataMapper team. Use at your own risk.

If you use this gem and find any issues, please let me know. Also, if you use this gem and things seem to be working just fine, then please let me know that too. Good luck!

DataMapper-0.3.0.1.gem

Horrible bug in DataMapper 0.3.0

Wednesday, April 9th, 2008

http://wm.lighthouseapp.com/projects/4819-datamapper/tickets/185-including-comparable-in-class-class-breaks-test-unit-and-probably-more#ticket-185-8

In the gem at the bottom of lib/data_mapper/support/typed_set.rb there is the following code:

class Class
  include Comparable
  def <=>(other)
    name <=> other.name
  end
end

This causese Test::Runner to through up an error similar to this:

/usr/local/lib/ruby/1.8/test/unit/collector/objectspace.rb:25:in `collect': undefined method `suite' for Gem::LoadError:Class (NoMethodError)
from /usr/local/lib/ruby/1.8/test/unit/collector/objectspace.rb:23:in `each_object'
from /usr/local/lib/ruby/1.8/test/unit/collector/objectspace.rb:23:in `collect'
from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:58
from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:213:in `[]'
from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:213:in `run'
from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:12:in `run'
from /usr/local/lib/ruby/1.8/test/unit.rb:278
from /usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake/rake_test_loader.rb:5
rake aborted!

It also causes your tests to blow up and not run. Which, if you’re trying to do any development causes some real problems! If you comment out ‘include Comparable’ from Class things seem to work just fine. I say seem, because I haven’t done any real extensive testing with this. The other thing you can do is revert to 0.2.5, but that’s up to you. Either way, it’s not really optimal. Let’s hope they fix this soon.

Let’s talk DataMapper

Monday, April 7th, 2008

As you may, or may not know, DataMapper is the new ORM framework on the scene these days in the Ruby world. It’s getting a lot of hype for being clean, fast, simple, and feature rich. Oh, and it’s not ActiveRecord. I think that seems to be the real thing that is driving people to DataMapper, the fact that it’s not ActiveRecord.

I will say DataMapper is clean, simple, and feature rich. In 0.3.0 they’ve added migrations, which is great. Btw, a little off topic, but I’m working on migration support for both ActiveRecord and DataMapper in Mack as we speak. It should, hopefully, be out sometime this week. What I won’t give DataMapper is that it’s fast. In my tests, and I’ll provide some number below, DataMapper only seems to win on inserts, after that ActiveRecord beats it hands down. In all fairness to DataMapper I’m starting to think that the problems are not at the DataMapper layer, but at the underlying Data Objects layer that DataMapper uses. As you’ll see from my tests DataMapper seems very heavily optimized towards MySQL over PostgreSQL. This, to me, leans towards a difference in the underlying adapters.

Another problem I have with DataMapper is that I have to set the ‘properties’ of the model inside the model itself. It’s an old school approach, and it does have the benefit of being self documenting, but it also has the drawbacks of constant maintenance and clutter at the top of your model. Not to mention potential conflicts when running through migrations, etc…

I’m also having one other little problem these days. This has only been a problem since I’ve gone DataMapper 0.3.0. At the end of some of rake tasks, if I have DataMapper required, I get this:

 

/usr/local/lib/ruby/1.8/test/unit/collector/objectspace.rb:25:in `collect': undefined method `suite' for Gem::VerificationError:Class (NoMethodError)
from /usr/local/lib/ruby/1.8/test/unit/collector/objectspace.rb:23:in `each_object'
from /usr/local/lib/ruby/1.8/test/unit/collector/objectspace.rb:23:in `collect'
from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:58
from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:213:in `[]'
from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:213:in `run'
from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:12:in `run'
from /usr/local/lib/ruby/1.8/test/unit.rb:278

The rake task completed successfully, but I get this fairly random error message. If anyone out there is also getting this message, please let me know what it is. I’m open for ideas on this one.

Despite these issues I have with DataMapper, I’m going to keep striving to provide support for both it and ActiveRecord in Mack. I think it’s important to give people a choice and not force them to use the one I think is either easier to code for, or better. Both of which I’m not saying about ActiveRecord, but in technologies in general.

Anyway, enough of my blather, here are the test results I was speaking about:

 

Running time 1.693881 seconds. [MESSAGE]: DM: postgresql: Inserts
Running time 2.799189 seconds. [MESSAGE]: AR: postgresql: Inserts

Running time 1.368185 seconds. [MESSAGE]: DM: postgresql: Individual Reads
Running time 0.734143 seconds. [MESSAGE]: AR: postgresql: Individual Reads

Running time 0.917551 seconds. [MESSAGE]: DM: postgresql: Bulk Reads
Running time 0.121198 seconds. [MESSAGE]: AR: postgresql: Bulk Reads

Running time 2.309244 seconds. [MESSAGE]: DM: postgresql: Updates
Running time 2.079578 seconds. [MESSAGE]: AR: postgresql: Updates

Running time 1.802914 seconds. [MESSAGE]: DM: postgresql: Deletes
Running time 1.708714 seconds. [MESSAGE]: AR: postgresql: Deletes

Running time 0.433761 seconds. [MESSAGE]: DM: mysql: Inserts
Running time 2.621093 seconds. [MESSAGE]: AR: mysql: Inserts

Running time 1.073741 seconds. [MESSAGE]: DM: mysql: Individual Reads
Running time 0.207305 seconds. [MESSAGE]: AR: mysql: Individual Reads

Running time 0.827842 seconds. [MESSAGE]: DM: mysql: Bulk Reads
Running time 0.073593 seconds. [MESSAGE]: AR: mysql: Bulk Reads

Running time 1.204845 seconds. [MESSAGE]: DM: mysql: Updates
Running time 1.738602 seconds. [MESSAGE]: AR: mysql: Updates

Running time 1.010774 seconds. [MESSAGE]: DM: mysql: Deletes
Running time 1.251691 seconds. [MESSAGE]: AR: mysql: Deletes

 

Â