Integration Testing with xUnit

A few years back, I had given up on xUnit in favor of Fixie because of the flexibility that Fixie provides. The xUnit project is highly opinionated, and geared strictly towards unit tests. It's great for that.

A broader testing strategy includes much more than just unit tests. With Fixie, I can implement any of the XUnit Test Patterns to implement a comprehensive automated test strategy (rather than, say, having different test frameworks for different kinds of tests).

In unit tests, each test method is highly isolated. In integration tests, this is usually not the case. Integration tests usually "touch" a lot more than a single class, and almost always, interact with other processes, files, and I/O. Unit tests are in-process, integration tests are out-of-process.

We can write our integration tests like our unit tests, but it's not always advantageous to do so because:

• Shared state (database)
• Expensive initialization

A typical integration test

If we look at a "normal" integration test we'd write on a more or less real-world project, its code would look something like:

• Set up data through the back door
• Set up data through the front door
• Build inputs
• Send inputs to system
• Verify direct outputs
• Verify side effects

One very simple example looks something like:

[Fact]
public async Task Should_edit_student()
{
    var createCommand = new Create.Command
    {
        FirstMidName = "Joe",
        LastName = "Schmoe",
        EnrollmentDate = DateTime.Today
    };

    var studentId = await SendAsync(createCommand);

    var editCommand = new Edit.Command
    {
        Id = studentId,
        FirstMidName = "Mary",
        LastName = "Smith",
        EnrollmentDate = DateTime.Today.AddYears(-1)
    };

    await SendAsync(editCommand);

    var student = await FindAsync<Student>(studentId);

    student.ShouldNotBeNull();
    student.FirstMidName.ShouldBe(editCommand.FirstMidName);
    student.LastName.ShouldBe(editCommand.LastName);
    student.EnrollmentDate.ShouldBe(editCommand.EnrollmentDate.GetValueOrDefault());
}

We're trying to test "editing", but we're doing it through the commands actually used by the application. In a real app, ASP.NET Core would modelbind HTTP request parameters to the Edit.Command object. I don't care to test with modelbinding/HTTP, so we go one layer below - send the command down, and test the result.

To do so, we need some setup, namely an original record to edit. The first set of code there does this through the front door, by sending the original "Create" command down.

From here on out, each awaited action is in its own individual transaction, mimicking as much as possible how these interactions would occur in the real world.

But with these styles of tests, there comes a couple of problems:

• Some setup I only want to do once, for all tests, similar to the real world
• Assertions are more complicated as these interactions can have many side effects

The first problem can be straightforward, but in the second, I usually tackle by switching my tests to a different pattern - "Testcase Class per Fixture". This lets me have a common setup with multiple test methods that each have different specific assertions.

With this in mind, how might we address both issues, with xUnit?

Shared Fixture Design

Each of these issues basically comes down to sharing context between tests. And there are a few ways to do these in xUnit - with collection fixtures and class fixtures.

Collection fixtures allow me to share context amongst many tests. Class fixtures allow me to share context in a class. The lifecycle of each determines what fixture I use for when:

• Collection fixtures are set up once per collection
• Class fixtures are set up once per test class
• Constructors/lifetimes are set up once per test method

That last one is important - if I do set up in an xUnit constructor or IAsyncLifetime on a test class, it executes once per test method - probably not what I want! What I'm looking for looks something like:

In each class, the Fixture contains the "Arrange/Act" parts of the test, and each test method contains the "Assert" part. That way, our test method names can describe the assertion from a behavioral perspective.

For our shared context, we'd want to create a collection fixture. This could include:

• Database stuff
• App stuff
• Configuration stuff
• Container stuff

Anything that gets set up in your application's startup is a good candidate for our shared context. Here's one example of a collection definition that uses both ASP.NET Core hosting stuff and Mongo2Go

public class SharedFixture : IAsyncLifetime
{
    private MongoDbRunner _runner;
    public MongoClient Client { get; private set; }
    public IMongoDatabase Database { get; private set; }

    public async Task InitializeAsync() {
        _runner = MongoDbRunner.Start();
        Client = new MongoClient(_runner.ConnectionString);
        Database = Client.GetDatabase("db");
        var hostBuilder = Program.CreateWebHostBuilder(new string[0]);
        var host = hostBuilder.Build();
        ServiceProvider = host.Services;
    }

    public Task DisposeAsync() {
        _runner?.Dispose();
        _runner = null;
        return Task.CompletedTask;
    }
}

Then we create a collection fixture definition in a separate class:

[CollectionDefinition(nameof(SharedFixture))]
public class SharedFixtureCollection : ICollectionFixture<SharedFixture>
{ }

Now that we have a definition of a shared fixture, we can try to use it in test. But first, we need to build out the test class fixture.

Class Fixture Design

For a class fixture, we're doing Arrange/Act as part of its design. But that also means we'll need to use our collection fixture. Our class fixture needs to use our collection fixture, and xUnit supports this.

Here's an example of a class fixture, inside a test class:

public class MyTestClass_For_Some_Context
    : IClassFixture<MyTestClass_For_Some_Context.Fixture> {
    private readonly Fixture _fixture;
    public MyTestClass_For_Some_Context(Fixture fixture)
        => _fixture = fixture;
    
    [Collection(nameof(SharedFixture)]
    public class Fixture : IAsyncLifetime {
        private readonly SharedFixture _sharedFixture;
        public Fixture(SharedFixture sharedFixture)
            => _sharedFixture = sharedFixture;

        public async Task InitializeAsync() {
            // Arrange, Act
        }
        public Order Order { get; set; }
        public OtherContext Context { get; set; }
        // no need for DisposeAsync
    }

    [Fact]
    public void Should_have_one_behavior() {
       // Assert
    }

    [Fact]
    public void Should_have_other_behavior() {
       // Assert
    }
}

The general idea is that fixtures must be supplied via the constructor, so I have to create a bit of a nested doll here. The class fixture takes the shared fixture, then my test class takes the class fixture. I use the InitializeAsync method for the "Arrange/Act" part of my test, then capture any direct/indirect output on properties on my Fixture class.

Then, in each test method, I only have asserts which look at the values in the Fixture instance supplied in my test method.

With this setup, my "Arrange/Act" parts are only executed once per test class. Each test method can be then very explicit about the behavior to test (the test method name) and assert only one specific aspect.

It's Ugly

Writing tests this manner allows me to fit inside xUnit's lifecycle configuration - but it's super ugly. I have attributes with magic values (the collection name), marker interfaces, nested classes (though this was my doing) and in general a lot of hoops.

What I would like to do is just have an easy way to define global, shared state, and define a separate lifecycle for integration tests. I don't mind using the IAsyncLifetime part but it's a bit annoying to have to work through a separate fixture class to do so.

So while it's feasible to write tests in this way, I'd suggest avoiding it. Having global shared state is possible, but combining those with class fixtures is just too complicated.

Instead, either use a framework more suited for this style of tests (Fixie or any of the BDD-style libraries), or just combine all asserts into one single test method.