Better Unit Tests Without Setup Methods

Unit testing frameworks usually have setup methods that are run before each test. One way to use them is to create objects that are used in tests. But many unit testing books recommend not to use setup methods. I have read many books about unit testing and none of them have recommended to use them. Those books that have mentioned setup methods have recommended to avoid them.

My opinion is that setup methods shouldn’t be used because they make tests more difficult to understand. Here is an example with C# and MSTest how it is usually used:

[TestClass]
public class MyCarTestsWithSetup1
{
  private MyCar _myCar;
  
  [TestInitialize]
  public void MySetup()
  {
    var engine = new Engine("V8");
    var stereo = new Stereo("Sony");
    _myCar = new MyCar(engine, stereo);
  }
  
  [TestMethod]
  public void CarInitialized_NotRunning()
  {
    Assert.IsFalse(_myCar.IsRunning());
  }
  
  [TestMethod]
  public void CarStarted_IsRunning()
  {
    _myCar.Start();
    Assert.IsTrue(_myCar.IsRunning());
  }
}

TestInitialize in row 6 is the attribute that makes MySetup method called before each test in MSTest. MySetup is the setup method here. We can see MyCar‘s constructor needs two objects and it is not trivial. It even seems that it is good to use the setup method because it reduces duplication. Let’s see another version where setup method isn’t used:

[TestClass]
public class MyCarTestsNoSetup1
{
  [TestMethod]
  public void CarInitialized_NotRunning()
  {
    var engine = new Engine("V8");
    var stereo = new Stereo("Sony");
    var myCar = new MyCar(engine, stereo);
    Assert.IsFalse(myCar.IsRunning());
  }
  
  [TestMethod]
  public void CarStarted_IsRunning()
  {
    var engine = new Engine("V8");
    var stereo = new Stereo("Sony");
    var myCar = new MyCar(engine, stereo);
    myCar.Start();
    Assert.IsTrue(myCar.IsRunning());
  }
}

The first impression is that the one with the setup method was better, because in this version we have to create a MyCar object at the beginning of each test method. Some might think that it was so easy with setup method, because we didn’t have to call new MyCar at the beginning of each test method.

Use Helper Methods Instead of a Setup Method

I agree that MyCarTestsWithSetup1 might be better than MyCarTestsNoSetup1 because of duplication, and thus it violates the important DRY principle. But if we change to use helper methods to create MyCar, it changes the situation:

[TestClass]
public class MyCarTestsNoSetup2
{
  [TestMethod]
  public void CarInitialized_NotRunning()
  {
    var myCar = CreateMyCar();
    Assert.IsFalse(myCar.IsRunning());
  }
  
  [TestMethod]
  public void CarStarted_IsRunning()
  {
    var myCar = CreateMyCar();
    myCar.Start();
    Assert.IsTrue(myCar.IsRunning());
  }

  private static MyCar CreateMyCar()
  {
    var engine = new Engine("V8");
    var stereo = new Stereo("Sony");
    return new MyCar(engine, stereo);
  }
}

This makes more sense. With CreateMyCar helper method we fixed the duplication. Is it more readable than the one with the setup method? In my opinion, it is. When we read a test method, we can see that car is first created. With the setup method we had to know that there was some magical (setup) method that creates a car. Another benefit is that CreateMyCar has a clear return type MyCar where setup method doesn’t return anything (return type is void).

Testing Not Default Value

Usually when we write unit tests, our object under test (or class/system under test) varies from test to test. Maybe in many tests it is the same but mostly there is also tests where default value can’t be used. The setup method isn’t at its best in these situations:

[TestClass]
public class MyCarTestsWithSetup2
{
  private MyCar _myCar;

  [TestInitialize]
  public void MySetup()
  {
    var engine = new Engine("V8");
    var stereo = new Stereo("Sony");
    _myCar = new MyCar(engine, stereo);
  }

  [TestMethod]
  public void CarInitialized_NotRunning()
  {
    Assert.IsFalse(_myCar.IsRunning());
  }

  [TestMethod]
  public void CarStarted_IsRunning()
  {
    _myCar.Start();
    Assert.IsTrue(_myCar.IsRunning());
  }

  [TestMethod]
  public void CarWithBigEngine_MakesMuchNoise()
  {
    var bigEngine = new Engine("V16");
    var stereo = new Stereo("Sony");
    var myCar = new MyCar(bigEngine, stereo);
    myCar.Start();
    Assert.AreEqual(myCar.GetDecibels(), 140);
  }
}

Here we have added only CarWithBigEngine_MakesMuchNoise test method compared to MyCarTestsWithSetup1 at the very beginning. But now the new test method doesn’t use _myCar anymore which is used with the other two tests. We can’t use _myCar here, and it means it would be useless to run the setup method with the new test method. But still, the setup method is run before each test regardless if the test uses _myCar or not.

Another version of this could be the one where we create _myCarWithBigEngine in the setup method:

[TestClass]
public class MyCarTestsWithSetup3
{
  private MyCar _myCar;
  private MyCar _myCarWithBigEngine;

  [TestInitialize]
  public void MySetup()
  {
    var engine = new Engine("V8");
    var stereo = new Stereo("Sony");
    _myCar = new MyCar(engine, stereo);

    var bigEngine = new Engine("V16");
    _myCarWithBigEngine = new MyCar(bigEngine , stereo);
  }

  [TestMethod]
  public void CarInitialized_NotRunning()
  {
    Assert.IsFalse(_myCar.IsRunning());
  }

  [TestMethod]
  public void CarStarted_IsRunning()
  {
    _myCar.Start();
    Assert.IsTrue(_myCar.IsRunning());
  }

  [TestMethod]
  public void CarWithBigEngine_MakesMuchNoise()
  {
    _myCarWithBigEngine.Start();
    Assert.AreEqual(_myCarWithBigEngine.GetDecibels(), 140);
  }
}

Now we are not creating a “big car” in the test but in the setup method. But this is quite a waste and confusing to create both _myCar and _myCarWithBigEngine before every test method just in case if some test method would use one of them.

Here is a version without setup method:

[TestClass]
public class MyCarTestsNoSetup3
{
  [TestMethod]
  public void CarInitialized_NotRunning()
  {
    var myCar = CreateMyCar();
    Assert.IsFalse(myCar.IsRunning());
  }
  
  [TestMethod]
  public void CarStarted_IsRunning()
  {
    var myCar = CreateMyCar();
    myCar.Start();
    Assert.IsTrue(myCar.IsRunning());
  }

  private static MyCar CreateMyCar()
  {
    var engine = new Engine("V8");
    var stereo = new Stereo("Sony");
    return new MyCar(engine, stereo);
  }

  [TestMethod]
  public void CarWithBigEngine_MakesMuchNoise()
  {
    var myCar = CreateMyCarWithBigEngine();
    myCar.Start();
    Assert.AreEqual(myCar.GetDecibels(), 140);
  }

  private static MyCar CreateMyCarWithBigEngine()
  {
    var bigEngine = new Engine("V16");
    var stereo = new Stereo("Sony");
    return new MyCar(bigEngine, stereo);
  }
}

It is the same as MyCarTestsNoSetup2 but there is the same new CarWithBigEngine_MakesMuchNoise test method as was in the MyCarTestsWithSetup2 and MyCarTestsWithSetup3. The difference here is that there isn’t any unnecessary setup method call and we don’t have to worry or get confused about unused _myCar. New helper method CreateMyCarWithBigEngine has a readable name and we can understand what it returns. In many ways, this is better without the setup method.

When to Use Setup Methods?

My simplified answer is: don’t use them with unit tests. Maybe if the same object is used in every test method, then it is not a big crime to use setup method. But even then I wouldn’t recommend it.

But in “bigger” automated tests (integration, end-to-end, etc.) setup methods can be really useful. For example, a database transaction can be initialized in setup method (and then rollbacked automatically in teardown method). You can read about this from my earlier blog post: Automatic Rollback to Integration Tests in C#.

Conclusion

Hopefully, this blog post has cleared somehow the “bad parts” of setup methods. As I mentioned earlier, I have read many unit testing books that tell to avoid setup methods.

I will end this blog post with the quote from Roy Osherove’s excellent The Art of Unit Testing book (which I’ve listed to my favorite books).

Setup methods have limitations, which you can get around using simple helper methods:
– Setup methods can only help when you need to initialize things.
– Setup methods aren’t always the best candidates for duplication removal. Removing duplication isn’t always about creating and initializing new instances of objects. Sometimes it’s about removing duplication in assertion logic, calling out code in a specific way.
– Setup methods can’t have parameters or return values.
– Setup methods can’t be used as factory methods that return values. They’re run before the test executes, so they must be more generic in the way they work. Tests sometimes need to request specific things or call shared code with a parameter for the specific test (for example, retrieve an object and set its property to a specific value).
– Setup methods should only contain code that applies to all the tests in the current test class, or the method will be harder to read and understand.

Roy Osherove in The Art of Unit Testing