Glenn Gillen

The ruby (and in particular the rails) community has grown at a pretty rapid rate since I started using it. The smaller community carried with it many benefits for a newbie, fewer options and fewer opinions on how to do things. Today though new developers have to contend with outdated documentation and tutorials (although http://guides.rubyonrails.org/ and the official docs have gone to great effort at late to keep everything updated), a veritable smorgasbord of options for everything from database access to asset packaging, and an even greater number of vocal opinions on how you should be doing it all. When it comes to testing it would be easy to think that the jury has spoken; You should be using Cucumber, Webrat, and RSpec.

Improved readability

One of the benefits of cucumber is that suddenly tests aren't written in ruby or any other language any more they're in plain text, and readable English text at that! And who can't read English? Let's take an example from Jonas Nicklas (let me also point out, Jonas has done a great job at pointing out where I think a lot of cuke practitioners get it fundamentally wrong… go read it):

Scenario: Adding a subpage

  Given I am logged in

  Given a microsite with a home page

  When I press "Add subpage"

  And I fill in "Title" with "Gallery"

  And I press "Ok"

  Then I should see a document called "Gallery"

Contrast that to some plain ol' ruby and the standard Webrat syntax (using shoulda):

context "logged in user"

  should "be able to add a subpage" do

    visit homepage

    click "Add subpage"

    fill_in "Title", :with => "Gallery"

    click "Ok"

    assert_have_selector ".documents" do

      assert_contain("Gallery")

    end

  end

end

    

Or if you want to refactor that last assertion into a helper method/macro:

context "logged in user"

  should "be able to add a subpage" do

    visit homepage

    click "Add subpage"

    fill_in "Title", :with => "Gallery"

    click "Ok"

    assert_has_document("Gallery")

  end

end

Now go back and read the first cucumber example, and the final shoulda/webrat example. Do it again. Is there any appreciable difference in readability? I'm confident I could put the latter in front of my non-tech involved fiancé and she'd be able to tell me what it's trying to do, but I'll discuss if that is even important later. The simple fact is that any ruby developer is going to be able to derive intention just as easily from both snippets of code.

Layers of indirection

Like an accomplished magician, cucumbers real trick is to have you looking somewhere other than where the magic is really happening. Open up a feature file and you'll be greeted by something like:

Scenario: Concealing behind a wall of smoke and mirros

  When I ask cucumber to run a scenario

  And I add 5 to 2 for a contrived step

  Then the scenario should have completed

  And it should add up to 7

Great, you know the intention behind this particular feature. But where do we go to find the steps for this test? Well there is obviously the steps file(s), but in a large project there are probably a few so which one? Say we've found the right file, how do you find the line responsible? You can't simply do a search for "ask cucumber to run a scenario" because the actual step definitions probably look something like:

When /^I ask ([a-z]+) to run a ([a-z]+)$/ do |framework,granularity|

  # do stuff

end

When /^I add ([0-9]+) to ([0-9]+) for a contrived step$/ do |a,b|

  # do stuff

end

Then /^the ([a-z]+) should have completed$/ do |granularity|

  # do stuff

end

Then /^it should add up to ([0-9]+)$/ do |result|

  # do stuff

end

    

Basically unless you already know the signature of the various methods you've got two options: 1) Read the whole damn file or 2) Run the features with verbose output so you can see which line gets invoked in your step file(s).

It's not an insurmountable problem, but it does present an additional barrier to understanding the code. And as I pointed out in the previous section, for a negligible gain given you can write plain ruby with the intention being just as clear. The benefit of that approach is that the plain ol' ruby isn't just your intention, it's actually the code that gets executed. There isn't an additional layer of comprehension required to see where the work is getting done, and if I've written a custom method/matcher/macro then I can actually search for it because it's immediately obvious what the name of that method is.

It's not actually English!

And here's my real issue with all those people drowning themselves in the cucumber kool-aid: they're convinced that this extra layer makes the intention of the application clear, that "the business" has visibility of what the product can do, and everybody is on the same page with regards to scope. If any of those have been a problem for you or your team writing it all down in a prescribed faux-English format isn't going to magically solve your problems. You need to sit down with your client/"the business" and actually engage with them, stop listening to them tell you what they want, and become involved enough to extract what it is they actually need.

I think the worst possible situation you can be in is thinking that at some point clients will be able to write the cucumber features themselves and you can focus on just making them pass. You've now forced upon them a foreign and limited grammar in which they are going to try and define the scope of their problems, and you've taken yourself out of the loop of truly understanding what the real nature of what you're building for the client is. (I discussed this further in my post: Nobody cares what tools you use)

The end result is you've both compromised and come to a middle ground that is less than optimal for both parties… and it takes a particularly skilled and dedicated developer (and a lot of work) to stop that being reflected in the quality of the delivered product.

How to do it better

If you're a one person dev team, you need to seriously ask yourself if you would derive any benefit from the indirection added by using cucumber? Will it improve the maintainability of your code? Your development velocity? Your ability to engage with your clients? The quality of the product you deliver? If you're part of a larger company and it sounds like cucumber will fix a lot of problems you've been suffering, I'll follow this up with a post arguing that most of it can probably be fixed by refocussing on doing agile properly.