Cucumber Rant
Unless you have been living under a rock for a while, you should be aware that Cucumber is a hot topic in the Ruby community. Its usage has spread like wildfire in the recent years and it seems like everyone but me is using it. Good for them!
Countless blog posts, presentations, screencasts and discussion threads about Cucumber can be found online but, strangely enough for such a popular technology, it's actually fairly difficult to find anything bad written or said about it (I actually only found a gist).
I have always been baffled by the whole idea behind Cucumber. Using natural language? Really?
“But The Customer Writes the Features and that's Beauty of it!”
Yeah right, let's get back down to earth for a minute...
Most customers do not want to know about how the software works and they could not care less as to how you make it work. They pay big bucks for functional software and, like it or not, that's pretty much it. As the forementioned piece brilliantly puts it: customers are good at creating problems and developers are allegedly good at solving them.
Asking your customer to describe some sort of business logic (granted they do actually have one) in a formal language would be welcomed, at the very best, with an embarassed grin.
But ok, let's imagine that your customer is willing to take some of his precious time to formalize the system he's himself paying for. Call me a pessimist/elitist/misanthrope but what are the odds that the file you will end up with will make sense, let alone be syntaxically valid?
"Enough reductio ad absurdum!", I hear you say. Of course, it's the software engineer's duty to translate the customer specifications into cukes. Then why bothering writing a formal specifications in plain english? The readability argument doesn't stand because even if your feature description reads like a beautiful haiku, you're left with a pile of step definitions which are a major pain to read.
One Step Too Far
Before settling for the much simpler RSpec and Webrat/Capybara approach, I have tried to get started with Cucumber. I painfully wrote a few scenarios and eventually looked at some examples of full featured Cucumber test suites on GitHub. And boy, what a mess this is!
Some annoyances which come to mind:
- Looking at a feature description, it can be difficult to tell what is part of the grammar and what is a variable
- The definition of steps looks terrible: it's a mess of strings, regular expressions and short chunks of actual code
- The actual end-to-end test implementation is scattered accross the steps, it's big a bowl of boilerplate code, pre and postconditions, assertions, ... this is hard to follow and a guaranteed maintenance nightmare
- More advanced use of the Cucumber DSL (like tables) look horrible unless you're having a strong nostalgic bent for ASCII art...
Automated Testing is Still a Developers Realm
Ruby developers are blessed with an awesome language which allows to express complex ideas in a very terse and readable way. We don't need another layer of abstraction, Ruby itself offers all the syntactic sugar and modularity one needs to write clean acceptance tests.
Maintaining a clean test environment requires a lot of discipline and, from a less subjective standpoint, automated acceptance testing represents a considerable engineering effort. In that regard, I can't imagine how adding an additional abstraction layer can bring any sort of value. It just goes against the KISS principle.
I've been avidly practicing BDD for quite some time now (my latest 15kLOC Rails project has way over 90% code coverage) and I don't see how using Cucumber would have made my development and testing any easier. On the contrary, I'm quite sure it would have made my whole acceptance test suite less readable, less maintainable, not to mention an awful lot slower.
To wrap this up, let's see what History has to tell us:
One of the design goals of COBOL was for COBOL code to be readable and understandable to non-programmers such as managers, supervisors and users. — Wikipedia
It didn't quite end up like that, did it?
