Enrico Campidoglio - AutoFixture

General-purpose customizations with AutoFixture

Tue, 16 Apr 2013 00:00:00 +0200

If you’ve been using AutoFixture in your tests for more than a while, chances are you’ve already come across the concept of customizations. If you’re not familiar with it, let me give you a quick introduction:

A customization is a group of settings that, when applied to a given Fixture object, control the way AutoFixture will create instances for the types requested through that Fixture.

At this point you might find yourself feeling an irresistible urge to know everything there’s to know about customizations. If that’s the case, don’t worry. There are a few resources online where you learn more about them. For example, I wrote about how to take advantage of customizations to group together test data related to specific scenarios.

In this post I’m going to talk about something different which, in a sense, is quite the opposite of that: how to write general-purpose customizations.

A (user) story about cooking

It’s hard to talk about test data without a bit of context. So, for the sake of this post, I thought we would pretend to be working on a somewhat realistic project. The system we’re going to build is an online catalogue of food recipies. The domain, at the very basic level, consists of three concepts:

Cookbook
Recipes
Ingredients

Now, let’s imagine that in our backlog of requirements we have one where the user wishes to be able to search for recepies that contain a specific set of ingredients. Or in other words:

As a foodie, I want to know which recipes I can prepare with the ingredients I have, so that I can get the best value for my groceries.

From the tests…

As usual, we start out by translating the requirement at hand into a set of acceptance tests. In order do that, we need to tell AutoFixture how we’d like the test data for our domain model to be generated.

For this particular scenario, we need every Ingredient created in the test fixture to be randomly chosen from a fixed pool of objects. That way we can ensure that all recepies in the cookbook will be made up of the same set of ingredients.

Here’s how such a customization would look like:

public class RandomIngredientsFromFixedSequence : ICustomization
{
    private readonly Random randomizer = new Random();
    private IEnumerable<Ingredient> sequence;

    public void Customize(IFixture fixture)
    {
        InitializeIngredientSequence(fixture);
        fixture.Register(PickRandomIngredientFromSequence);
    }

    private void InitializeIngredientSequence(IFixture fixture)
    {
        this.sequence = fixture.CreateMany<Ingredient>();
    }

    private Ingredient PickRandomIngredientFromSequence()
    {
        var randomIndex = this.randomizer.Next(0, sequence.Count());
        return sequence.ElementAt(randomIndex);
    }
}

Here we’re creating a pool of ingredients and telling AutoFixture to randomly pick one of those every time it needs to create an Ingredient object by using the Fixture.Register method.

Since we’ll be using Xunit as our test runner, you can take advantage of the AutoFixture Data Theories to keep our tests succinct by using AutoFixture in a declarative fashion. In order to do so, we need to write an xUnit Data Theory attribute that tells AutoFixture to use our new customization:

public class CookbookAutoDataAttribute : AutoDataAttribute
{
    public CookbookAutoDataAttribute()
        : base(new Fixture().Customize(
                   new RandomIngredientsFromFixedSequence())))
    {
    }
}

If you prefer to use AutoFixture directly in your tests, the imperative equivalent of the above is:

var fixture = new Fixture();
fixture.Customize(new RandomIngredientsFromFixedSequence());

At this point, we can finally start writing the acceptance tests to satisfy our original requirement:

public class When_searching_for_recipies_by_ingredients
{
    [Theory, CookbookAutoData]
    public void Should_only_return_recipes_with_a_specific_ingredient(
        Cookbook sut,
        Ingredient ingredient)
    {
        // When
        var recipes = sut.FindRecipies(ingredient);
        // Then
        Assert.True(recipes.All(r => r.Ingredients.Contains(ingredient)));
    }

    [Theory, CookbookAutoData]
    public void Should_include_new_recipes_with_a_specific_ingredient(
        Cookbook sut,
        Ingredient ingredient,
        Recipe recipeWithIngredient)
    {
        // Given
        sut.AddRecipe(recipeWithIngredient);
        // When
        var recipes = sut.FindRecipies(ingredient);
        // Then
        Assert.Contains(recipeWithIngredient, recipes);
    }
}

Notice that during these tests AutoFixture will have to create Ingredient objects in a couple of different ways:

indirectly when constructing Recipe objects associated to a Cookbook
directly when providing arguments for the test parameters

As far as AutoFixture is concerned, it doesn’t really matter which code path leads to the creation of ingredients. The algorithm provided by the RandomIngredientsFromFixedSequence customization will apply in all situations.

…to the implementation

After a couple of Red-Green-Refactor cycles spawned from the above tests, it’s not completely unlikely that we might end up with some production code similar to this:

// Cookbook.cs
public class Cookbook
{
    private readonly ICollection<Recipe> recipes;

    public Cookbook(IEnumerable<Recipe> recipes)
    {
        this.recipes = new List<Recipe>(recipes);
    }

    public IEnumerable<Recipe> FindRecipies(params Ingredient[] ingredients)
    {
        return recipes.Where(r => r.Ingredients.Intersect(ingredients).Any());
    }

    public void AddRecipe(Recipe recipe)
    {
        this.recipes.Add(recipe);
    }
}

// Recipe.cs
public class Recipe
{
    public readonly IEnumerable<Ingredient> Ingredients;

    public Recipe(IEnumerable<Ingredient> ingredients)
    {
        this.Ingredients = ingredients;
    }
}

// Ingredient.cs
public class Ingredient
{
    public readonly string Name;

    public Ingredient(string name)
    {
        this.Name = name;
    }
}

Nice and simple. But let’s not stop here. It’s time to take it a bit further.

An opportunity for generalization

Given the fact that we started working from a very concrete requirement, it’s only natural that the RandomIngredientsFromFixedSequence customization we came up at with encapsulates a behavior that is specific to the scenario at hand. However, if we take a closer look we might notice the following:

The only part of the algorithm that is specific to the original scenario is the type of the objects being created. The rest can easily be applied whenever you want to create objects that are picked at random from a predefined pool.

An opportunity for writing a general-purpose customization has just presented itself. We can’t let it slip.

Let’s see what happens if we extract the Ingredient type into a generic argument and remove all references to the word “ingredient”:

public class RandomFromFixedSequence<T> : ICustomization
{
    private readonly Random randomizer = new Random();
    private IEnumerable<T> sequence;

    public void Customize(IFixture fixture)
    {
        InitializeSequence(fixture);
        fixture.Register(PickRandomItemFromSequence);
    }

    private void InitializeSequence(IFixture fixture)
    {
        this.sequence = fixture.CreateMany<T>();
    }

    private T PickRandomItemFromSequence()
    {
        var randomIndex = this.randomizer.Next(0, sequence.Count());
        return sequence.ElementAt(randomIndex);
    }
}

Voilà. We just turned our scenario-specific customization into a pluggable algorithm that changes the way objects of any type are going to be generated by AutoFixture. In this case the algorithm will create items by picking them at random from a fixed sequence of T.

The CookbookAutoDataAttribute can easily changed to use the general-purpose version of the customization by closing the generic argument with the Ingredient type:

public class CookbookAutoDataAttribute : AutoDataAttribute
{
    public CookbookAutoDataAttribute()
        : base(new Fixture().Customize(
                   new RandomFromFixedSequence<Ingredient>())))
    {
    }
}

The same is true if you’re using AutoFixture imperatively:

var fixture = new Fixture();
fixture.Customize(new RandomFromFixedSequence<Ingredient>());

Wrapping up

As I said before, customizations are a great way to set up test data for a specific scenario. Sometimes these configurations turn out to be useful in more than just one situation.

When such opportunity arises, it’s often a good idea to separate out the parts that are specific to a particular context and turn them into parameters. This allows the customization to become a reusable strategy for controlling AutoFixture’s behavior across entire test suites.

Keep your unit tests DRY with AutoFixture Customizations

Thu, 15 Dec 2011 00:00:00 +0100

When I first incorporated AutoFixture as part of my daily unit testing workflow, I noticed how a consistent usage pattern had started to emerge. This pattern can be roughly summarized in three steps:

Initialize an instance of the Fixture class.
Configure the way different types of objects involved in the test should be created by using the Build method.
Create the actual objects with the CreateAnonymous or CreateMany methods.

As a result, my unit tests had started to look a lot like this:

[Test]
public void WhenGettingAListOfPublishedPostsThenItShouldOnlyIncludeThose()
{
    // Step 1: Initialize the Fixture
    var fixture = new Fixture();

    // Step 2: Configure the object creation
    var draft = fixture.Build()
        .With(a => a.IsDraft = true)
        .CreateAnonymous();
    var publishedPost = fixture.Build()
        .With(a => a.IsDraft = false)
        .CreateAnonymous();
    fixture.Register(() => new[] { draft, publishedPost });

    // Step 3: Create the anonymous objects
    var posts = fixture.CreateMany();

   // Act and Assert...
}

In this particular configuration, AutoFixture will satisfy all requests for IEnumerable types by returning the same array with exactly two Post objects: one with the IsDraft property set to True and one with the same property set to False.

At that point I felt pretty satisfied with the way things were shaping up: I had managed to replace entire blocks of boring object initialization code with a couple of calls to the AutoFixture API, my unit tests were getting smaller and all was good.

Duplication creeps in

After a while though, the configuration lines created in Step 2 started to repeat themselves across multiple unit tests. This was naturally due to the fact that different unit tests sometimes shared a common set of object states in their test scenario. Things weren’t so DRY anymore and suddenly it wasn’t uncommon to find code like this in the test suite:

[Test]
public void WhenGettingAListOfPublishedPostsThenItShouldOnlyIncludeThose()
{
    var fixture = new Fixture();
    var draft = fixture.Build()
        .With(a => a.IsDraft = true)
        .CreateAnonymous();
    var publishedPost = fixture.Build()
        .With(a => a.IsDraft = false)
        .CreateAnonymous();
    fixture.Register(() => new[] { draft, publishedPost });
    var posts = fixture.CreateMany();

    // Act and Assert...
}

[Test]
public void WhenGettingAListOfDraftsThenItShouldOnlyIncludeThose()
{
    var fixture = new Fixture();
    var draft = fixture.Build()
        .With(a => a.IsDraft = true)
        .CreateAnonymous();
    var publishedPost = fixture.Build()
        .With(a => a.IsDraft = false)
        .CreateAnonymous();
    fixture.Register(() => new[] { draft, publishedPost });
    var posts = fixture.CreateMany();

    // Different Act and Assert...
}

See how these two tests share the same initial state even though they verify completely different behaviors? Such blatant duplication in the test code is a problem, since it inhibits the ability to make changes. Luckily a solution was just around the corner as I discovered customizations.

Customizing your way out

A customization is a pretty general term. However, put in the context of AutoFixture it assumes a specific definition:

A customization is a group of settings that, when applied to a given Fixture, control the way AutoFixture will create anonymous instances of the types requested through that Fixture.

What that means is that I could take all the boilerplate configuration code produced during Step 2 and move it out of my unit tests into a single place, that is a customization. That allowed me to specify only once how different objects needed to be created for a given scenario, and reuse that across multiple tests.

public class MixedDraftsAndPublishedPostsCustomization : ICustomization
{
    public void Customize(IFixture fixture)
    {
        var draft = fixture.Build()
            .With(a => a.IsDraft = true)
            .CreateAnonymous();
        var publishedPost = fixture.Build()
            .With(a => a.IsDraft = false)
            .CreateAnonymous();
        fixture.Register(() => new[] { draft, publishedPost });
    }
}

As you can see, ICustomization is nothing more than a role interface that describes how a Fixture should be set up. In order to apply a customization to a specific Fixture instance, you’ll simply have to call the Fixture.Customize(ICustomization) method, like shown in the example below. This newly won encapsulation allowed me to rewrite my unit tests in a much more terse way:

[Test]
public void WhenGettingAListOfDraftsThenItShouldOnlyIncludeThose()
{
    // Step 1: Initialize the Fixture
    var fixture = new Fixture();

    // Step 2: Apply the customization for the test scenario
    fixture.Customize(new MixedDraftsAndPublishedPostsCustomization());

    // Step 3: Create the anonymous objects
    var posts = fixture.CreateMany();

    // Act and Assert...
}

The configuration logic now exists only in one place, namely a class whose name clearly describes the kind of test data it will produce. If applied consistently, this approach will in time build up a library of customizations, each representative of a given situation or scenario. Assuming that they are created at the proper level of granularity, these customizations could even be composed to form more complex scenarios.

Conclusion

Customizations in AutoFixture are a pretty powerful concept in of themselves, but they become even more effective when mapped directly to test scenarios. In fact, they represent a natural place to specify which objects are involved in a given scenario and the state they are supposed to be in. You can use them to remove duplication in your test code and, in time, build up a library of self-documenting modules, which describe the different contexts in which the system’s behavior is being verified.

Behavior changes in AutoFixture 2.2 – Anonymous numbers

Tue, 06 Sep 2011 00:00:00 +0200

Now that AutoFixture 2.2 is approaching on the horizon, it’s a good time to start talking about some of the changes that were made to the underlying behavior of some existing APIs. I’ll start off this series of posts by focusing on the new generation strategy for anonymous numbers.

The good old fashioned way

Before I jump into the details of what exactly has been changed and how, allow me to set up a little bit of stage:

A key part of AutoFixture’s mission statement is to make the process of authoring unit tests faster by providing an easy way of creating test values (or “specimens“) for the variables involved in the test. The goal of providing values that are as neutral as possible to the test scenario at hand is achieved by employing “constrained non-deterministic” generation algorithms.

Put in simple terms, this essentially means that AutoFixture will come up with test values at run time that can be considered “random” within some predefined bounds. These bounds are imposed at the lowest level by the variable’s own data type: a string is a string, a number is a number and so on. More constraints, however, can be added at a higher level, based on any semantics the variable may have in the specific test scenario. For example a string can’t be longer than 20 characters or a number must be between 1 and 100.

AutoFixture comes with a set of built-in generation algorithms that can produce test values for all the primitive types included in the .NET Framework. The algorithm for numeric types has historically been based on individually incremented sequences, one for each numeric data type. Let’s look at an example that illustrates this:

var fixture = new Fixture();
Console.WriteLine("Byte specimen is {0}, {1}",
    fixture.CreateAnonymous(),
    fixture.CreateAnonymous());
Console.WriteLine("Int32 specimen is {0}, {1}",
    fixture.CreateAnonymous(),
    fixture.CreateAnonymous());
Console.WriteLine("Single specimen is {0}, {1}",
    fixture.CreateAnonymous(),
    fixture.CreateAnonymous());

// The output will be:
// Byte specimen is 1, 2
// Int32 specimen is 1, 2
// Single specimen is 1, 2

The key point here is that AutoFixture will only guarantee unique numeric specimens within the scope of a specific data type. Now, you may wonder how this would be a problem. Well, it certainly isn’t in itself, but if you asked AutoFixture to give you an anonymous instance of a class with multiple properties of different numeric types, you would get something like this:

public class NumericBag
{
    public byte ByteValue { get; set; }
    public int Int32Value { get; set; }
    public float SingleValue { get; set; }
}

var fixture = new Fixture();
var specimen = fixture.CreateAnonymous();
Console.WriteLine("ByteValue property is {0}", specimen.ByteValue);
Console.WriteLine("Int32Value property is {0}", specimen.Int32Value);
Console.WriteLine("SingleValue property is {0}", specimen.SingleValue);

// The output will be:
// ByteValue property is 1
// Int32Value property is 1
// SingleValue property is 1

We can agree that the end result doesn’t exactly live up to the expectation of anonymous values being “random”. Starting from version 2.2, however, this behavior is due to change.

The fresh new way

AutoFixture has taken a different approach to numeric specimen generation and will now by default return unique values across all numeric types. Running our first example in AutoFixture 2.2 will therefore yield a very different result:

var fixture = new Fixture();
Console.WriteLine("Byte specimen is {0}, {1}",
    fixture.CreateAnonymous(),
    fixture.CreateAnonymous());
Console.WriteLine("Int32 specimen is {0}, {1}",
    fixture.CreateAnonymous(),
    fixture.CreateAnonymous());
Console.WriteLine("Single specimen is {0}, {1}",
    fixture.CreateAnonymous(),
    fixture.CreateAnonymous());

// The output will be:
// Byte specimen is 1, 2
// Int32 specimen is 3, 4
// Single specimen is 5, 6

In other words, AutoFixture is being a little more “non-deterministic” when it comes to numeric test values. Take for example the following scenario:

public class NumericBag
{
    public byte ByteValue { get; set; }
    public int Int32Value { get; set; }
    public float SingleValue { get; set; }
}

var fixture = new Fixture();
var specimen = fixture.CreateAnonymous();
Console.WriteLine("ByteValue property is {0}", specimen.ByteValue);
Console.WriteLine("Int32Value property is {0}", specimen.Int32Value);
Console.WriteLine("SingleValue property is {0}", specimen.SingleValue);

// The output will be:
// ByteValue property is 1
// Int32Value property is 2
// SingleValue property is 3

See how all the numeric properties on the generated object have different values? That’s what I’m talking about.

Now, in theory, this shouldn’t be considered a breaking change. I say this because AutoFixture is all about anonymous variables, which, by definition, can’t be expected to have specific values during a test run. So, as long as you’ve played by this rule, the new behavior shouldn’t impact any of your existing tests.

However, if this does turn out to be a problem or you simply prefer the old way of doing things, you shouldn’t feel left out in the cold. The previous behavior is still in the box, packaged up in a nice customization unambiguously named NumericSequencePerTypeCustomization. The simple act of adding it to a Fixture instance will restore things the way they were:

var fixture = new Fixture();
fixture.Customize(new NumericSequencePerTypeCustomization());

If you wish to try this out today, I encourage you to go head and grab the latest build off of AutoFixture’s project page on TeamCity. Enjoy.

Anonymous delegates in AutoFixture

Mon, 01 Aug 2011 00:00:00 +0200

I’m excited to announce that AutoFixture now officially supports delegates in the main trunk up on CodePlex.

If you aren’t familiar with AutoFixture, let me give you the pitch:

AutoFixture is an open source framework for .NET designed to minimize the ‘Arrange’ phase of your unit tests. Its primary goal is to allow developers to focus on what is being tested rather than how to setup the test scenario, by making it easier to create object graphs containing test data.

Does this sound interesting to you? In that case head over to the AutoFixture CodePlex site and find out more. You’ll be glad you did.

For those of you already familiar with AutoFixture, the newly added support for delegates means that every time AutoFixture is asked to create an anonymous instance of a delegate type (or more precisely a delegate specimen), it will actually return one, instead of throwing an exception.

So, you’ll be able to say things like:

public delegate void MyDelegate();

var fixture = new Fixture();
var delegateSpecimen = fixture.CreateAnonymous();

and get back a delegate pointing to a dynamically generated method, whose signature matches the one of the requested delegate type. In other words AutoFixture will satisfy the requests for delegates by providing a method specimen.

That’s cool, but it may leave you wondering: what on Earth does a method specimen do when it gets invoked? Well, in order to answer that question, we need to look at the signature of the delegate that was requested in the first place. The rule basically says:

If the signature of the requested delegate has a return value (i.e. it’s a function), the method specimen will always return an anonymous value of the return type.
If the signature of the requested delegate doesn’t have a return value (i.e. it’s an action) the returned method specimen will have an empty body.

This principle is best illustrated by examples. Consider the following code snippet:

var fixture = new Fixture();
var funcSpecimen = fixture.CreateAnonymous();
var result = funcSpecimen();

// result = "fd95320f-0a37-42be-bd49-3afbbe089d9d"

In this example, since the signature of the requested delegate has a return value of type String, the result variable will contain an anonymous string value, which in AutoFixture usually translates into a GUID. On the other hand, if requested delegate didn’t have a return value, invoking the anonymous delegate would do just about nothing:

var fixture = new Fixture();
var actionSpecimen = fixture.CreateAnonymous();
actionSpecimen("whatever"); // no-op

Note that in both cases any input arguments passed to the anonymous delegate will be ignored, since they don’t have any impact on the generated method specimen.

Now, if you’re using AutoFixture from its NuGet package (which, by the way, you should) you’ll have to wait until the next release to get this feature. However, taking advantage of it with the current version of AutoFixture requires a minimal amount of effort. Just grab the DelegateGenerator.cs class from AutoFixture’s main trunk on CodePlex and include it in your project. You’ll then be able to add support for delegates to your Fixture instance by simply saying:

var fixture = new Fixture();
fixture.Customizations.Add(new DelegateGenerator());

You can even wrap that up in a Customization to make it more centralized and keep your test library DRY:

public class DelegateCustomization : ICustomization
{
    public void Customize(IFixture fixture)
    {
        if (fixture == null)
        {
            throw new ArgumentNullException("fixture");
        }

        fixture.Customizations.Add(new DelegateGenerator());
    }
}

Before finishing this off, let me give you a more concrete example that shows how this is useful in a real world scenario. Keeping in mind that delegates offer a pretty terse way to implement the Strategy Design Pattern in .NET, consider this implementation of the IEqualityComparer interface:

public class EqualityComparer : IEqualityComparer
{
    private readonly Func equalityStrategy;
    private readonly Func hashCodeStrategy;

    public EqualityComparer(Func equalityStrategy, Func hashCodeStrategy)
    {
        if (equalityStrategy == null)
        {
            throw new ArgumentNullException("equalityStrategy");
        }

        if (hashCodeStrategy == null)
        {
            throw new ArgumentNullException("hashCodeStrategy");
        }

        this.equalityStrategy = equalityStrategy;
        this.hashCodeStrategy = hashCodeStrategy;
    }

    public bool Equals(T x, T y)
    {
        return equalityStrategy(x, y);
    }

    public int GetHashCode(T obj)
    {
        return hashCodeStrategy(obj);
    }
}

That’s a nice flexible class that, by allowing to specify the comparison logic in the form of delegates, is suitable in different scenarios. Before the support for delegates was added, however, having AutoFixture play along with this class in the context of unit testing would be quite problematic. The tests would, in fact, fail consistently with a NotSupportedException, since the constructor of the EqualityComparer class requires the creation of two delegates. Luckily, this is not a problem anymore.