A blog about software development, written by Daniel Diekmeier. Archive.

At work, I'm thinking a lot about testing, and what kind of tests help me to understand and maintain projects. However, I keep coming across words like "correct" in test titles, and it has happened often enough that I'm finally writing this blog post.

Please stop.

Let's start with one example in a little more detail, but there will be more at the end. I'm talking about situations like this:

describe "#average_score"
  it "returns correct average" do
    # cool setup here ...
    expect(thing.average_score).to eq(4.2)
  end
end

My problem with this is: What the heck is "correct" supposed to mean? When we look at the code, we might find something like this:

def average_score
  ratings = users.only_public.filter_map(&:max_rating)
  return nil if ratings.empty?

  ratings.sum / ratings.size.to_f
end

This is adapted from actual production code I had to work with recently. I didn't write the original, but I had to change it and wanted to make sure I didn't break anything. Thankfully, we have a test that ensures the method's behaviour, it simply has to … eh … hm … be correct!

First of all: There are a bunch of different branches here, but only one test. Second of all: The test should, in plain terms, explain what we expect to happen.

Imagine you're sitting down with a colleague, showing them your code, and they ask a question:

Colleague: Hey, what's average_score doing?
You: It's returning the correct average.
Colleague: [backs away from your desk]

No! That's unhinged!

Colleague: Hey, what's average_score doing?
You: It's returning the average of users' max_rating.
Colleague: Yeah, it totally does!
You: And actually, we only count public users.
Colleague: That's awesome!

I want my tests to give the impression that the person who wrote them thought about what a human reading the tests probably wants to know. (Because I do think about that a lot!) To that end, I'd update the tests to look something like this:

describe "#average_score"
  it "returns average of users' max_rating" do
    # cool setup here ...
    expect(thing.average_score).to eq(4.2)
  end

  it "excludes users that are not public"
  it "excludes users that don't have a max_rating"
  it "returns nil if there are no ratings"
end

(I left out the test bodies for brevity, since they are not important to the point I'm trying to make.)

Some might argue that this is basically just a code comment and that it's coupled too closely to the underlying code. If people want to know the behaviour of the code, they should "simply" read it. To that I say: Smell my shorts. Your tests deserve to be understandable to humans. Humans deserve to understand your tests.

Here are some more example tests, adapted from real life, together with suggestions on rewriting their titles. Notice how the titles mention some nouns from their projects, but it's pretty much impossible to guess what the underlying logic is doing. "Correctly" is used as a placeholder for a real explanation, giving the impression that the title is somehow useful.

- it "returns the correct URLs"
+ it "returns URLs of visible items"

- it "strips the email address correctly"
+ it "strips spaces from the email address"

- it "returns correct number of items when given an infinite range"
+ it "returns all items after start of endless range"

- it "includes correct columns in csv"
+ it "includes 'id', 'name', 'sales' columns in csv"

# Same thing can happen with synonyms
# of "correct", like "right":

- it "returns the right prices"
+ it "returns the prices plus taxes for customer"

- it "has the right slug format"
+ it "has slug in kebab case without umlauts"

- it "creates the right documents"
+ it "creates only the German tax documents"

There is also the opposite case where "correctly" is not a placeholder for an explanation, but instead is contributing nothing. In these cases, you can skip it.

- it "correctly coerces string 'yes' to true" do
- it "correctly coerces numeric 1 to true" do
+ it "coerces 'yes' to true" do
+ it "coerces 1 to true" do

(Yes, thank you, I think I'll assume that the tests are already correct! It's actually a bit suspicious that you have to emphasise it this much.)

I hope you'll think of this post when you write your next tests or when you do the next code review of tests somebody else wrote. Remember: Do the correct thing!