ApprovalTests, One of My Favorite NuGet

Every C# developer has few favorite NuGets. I will introduce one of my favorite: ApprovalTests. It is a great tool for unit testing. In short, it makes asserting whole objects easier than to check all properties one at the time. If a test fails it will open a diff tool and shows the difference between actual and expected results. I found ApprovalTests from Pluralsight course when I was learning C#. Since that, I’ve used it continuously.

Show Me the Code!

Here is a really simple class to test:

public static class HelloApproval
{
public static string Hello()
{
return "Hello world";
}
}

And here is the NUnit test code:

[TestFixture]
[UseReporter(typeof(DiffReporter))]
public class HelloApprovalTests
{
[Test]
public void HelloTest_Traditional()
{
Assert.AreEqual("Hello world", HelloApproval.Hello());
}

[Test]
public void HelloTest_Approval()
{
Approvals.Verify(HelloApproval.Hello());
}
}

We have here traditional test (HelloTest_Traditional) and test using ApprovalTests (HelloTest_Approval). If you haven’t ever seen ApprovalTests, HelloTest_Approval doesn’t even seems to be a test: there isn’t any assertion. The trick is how it runs and what it does beyond the code.

Let’s run the tests. HelloTest_Traditional passes, but then Visual Studio opens your diff tool (I have Beyond Compare which I really like). When you close it the test is failed.

The first reaction might be “What is this and why did my diff tool open??”.

On the left is the actual output. As you see there is the output of Hello method. The file is named *.received.txt and this is the way ApprovalTests names actual output files. You should not save this file to your source control or Visual Studio project. Best practice is to put *.received.* to your ignore file.

On the right is the expected output. As you can see it is initially empty. Because actual output (“Hello world”) and expected output (empty) doesn’t match, the test failed. The file is named *.approved.txt and this is the way ApprovalTests names expected output files. Unlike received file on the left, you should save this file to your source control and Visual Studio project. Now copy the text on the left file to the file on the right and save it.

If you now look at the files in your project, after selecting show all files (1), you can see received.txt and approved.txt files (2). These are the files you just saw in your diff tool. Include the approved.txt file to your project (right click on it and choose “Include In Project”).

You have now saved what you expect: approved.txt file with the text “Hello World”. Now when you run tests again they will all pass and diff tool won’t open. ApprovalTests opens the diff tool only if the test fails. The reason why tests now passed is that now approved.txt (expected) and received.txt (actual) files’ contents are same.

If you look at the files again (after pressing refresh (1)), you will see that there isn’t any more received.txt file (2). This is how ApprovalTests works: if the test passes, it will not save received.txt file but delete it.

The Best Part: When Output Changes

Now we change our method a bit by changing ‘w’ to ‘W’ and put explanation mark:

public static string Hello()
{
return "Hello World!";
}

If we run HelloTest_Approval it will open the diff tool again and the test fails:

Here is the part I really love about ApprovalTests: if actual and expected outputs don’t match it opens the diff tool and you can edit the expected output (approved.txt file). As you can see, diff tool shows the difference illustratively. We can now just copy the text from left to right and save approved.txt file.

If we again look at the files, received.txt has appeared there again. The reason is that the test failed, and it will create the received.txt file. This is how ApprovalTests works: if a test fails, it will create a received.txt file, but if a test passes, it will delete a received.txt file.

If we now run the test again it will pass because we just updated approved.txt file. And like when the test passes, the received.txt file has been deleted.

Assert the Whole Object with One Assert

The previous example was just simplification to show how AprovalTests work. In my opinion, there isn’t any need to use ApprovalTests to assert single values. But it becomes powerful when asserting whole objects. Unfortunately, it doesn’t work as easy as you hope but it is not difficult. Here is our code/object to test:

public class Player
{
public string FirstName { get; set; }
public string LastName { get; set; }
}

And here is first test method:

[Test]
public void PlayerTest()
{
var player = new Player { FirstName = "Lassi", LastName = "Autio" };
Approvals.Verify(player);
}

If we run the test it will fail and opens the diff tool:

This doesn’t help. It uses ToString method to get an output of the object. You can implement ToString method to your class but that would be too cumbersome if you have many objects to compare.

My solution is to serialize the object to JSON formatted string. For example Newtonsoft.Json is a NuGet that can do it for you. Change the object to JSON formatted string first and then pass that string to Approvals.Verify:

string json = JsonConvert.SerializeObject(player);
Approvals.Verify(json);

Now if we run the test, the output in diff tool is better:

The output is now better, but it is in one row. If we had bigger object it would be difficult to figure out differences from this. The solution is to assert data as JSON with VerifyJson method:

Approvals.VerifyJson(json);

Now if we run the test, the output is really nice:

You can use even bigger objects and it will format it in JSON really nice. Practically when I use ApprovalTests I always use this Approvals.VerifyJson method.

Compare this with asserting every single property of the object, or even few of them. It is easier to use ApprovalTests.

Best with Legacy Code

ApprovalTests is powerful with legacy code (ie. code that doesn’t have tests). For example, if you have a method that returns an object, but there aren’t any tests. You can write the first test even if you don’t know what the method really should return: just run the method and verify the returned object with ApprovalTests. This approach assumes that current code works.

Now you have a test that at least tests something and more than just one property. One by one if you write tests like this you will have better test coverage. Then you can refactor code little by little to make it clearer because these tests will fail if some object will change.

Usually, these tests aren’t unit tests because legacy code isn’t designed according to TDD. Rather they are integration tests or tests that test many methods, not just one unit. But if your code doesn’t have tests, this is a good way to start with: first, write some integration tests and then you can refactor the code so that it is able to unit test.

Conclusion

ApprovalTests is a NuGet that can verify the whole object with one assert. If the test fails (object isn’t as expected) it will open your diff tool and show the differences. Then you can change your code or update the expected output.

There are good tutorials on Youtube. For example, Using ApprovalTests .Net 02 NUnit is a good video to start from. And then there are other tutorials where you can get more information about ApprovalTests.

ApprovalTests isn’t only for C#. For example, there is one for Java, C++, Python etc. You can find all from Github.

I really recommend you to become familiar with this if you are a C# developer.

Recap: How ApprovalTests Work

• If a test doesn’t pass:
  • Opens diff tool to show the difference between actual and expected output
  • Saves actual output to the received.txt file
  • You can update expected output to the approved.txt file with a diff tool
• If a test passes:
  • Doesn’t open diff tool, because there is no difference to show
  • Doesn’t save the actual output to the received.txt file
• Actual output: received.txt file
• Expected output: approved.txt file
• The first run practically always fails because there isn’t any approved.txt file yet

Tutorials and Other Material

• Using ApprovalTests in .Net on Youtube
• Approval Tests for .NET on Pluralsight
• Github

Ps. Doesn’t Work with .NET Core Yet

Unfortunately, ApprovalTests doesn’t work with .NET Core yet. There is an issue about this in Github and seems that they are working on it. Latest “we are working on it” update was on 2018-02-23.