End-to-End Integration Testing with NServiceBus

One of the interesting side effects of adding diagnostic events to infrastructure is that you can now "listen in" to what's going on in your applications for black box testing. This can be especially useful in scenarios where you're building on top of a framework that includes a lot of built-in behavior, such as ASP.NET Core and NServiceBus.

In ASP.NET Core, you can write tests directly against your controllers/handlers/pages, but that won't execute the entire request pipeline, only a little snippet. Similarly, you can write a test against an NServiceBus handler, but there's a lot more behavior going on around it.

To get an idea that the entire application can execute successfully, you need to actually run the application, which can be a huge challenge for message-oriented architectures that are by nature asynchronous.

To help with this, I created the NServiceBus.Extensions.IntegrationTesting project that leverages the diagnostics events I blogged about to listen in to messages sent/received.

Usage

I created a rather complex scenario in that series, in which an API call would result in either an orchestration or choreography-based workflow, depending on the API call I used. Eventually, this worklow terminates, but the only way I could test this entire flow was to run the entire system.

If I want to test the entire end-to-end system, I can start with the integration testing capabilities with ASP.NET Core. This package creates a special test host and test HTTP client that executes the entire pipeline, but all in-memory.

I want to do similar with NServiceBus, except using the Learning Transport instead of a "real" transport. That way, I don't have to worry about provisioning queues in test environments.

The NServiceBus integration test project includes two extensions:

• A helper method to configure your endpoint for integration testing
• A fixture class that observes the diagnostic events around some supplied action

In our Xunit tests, we create a derived WebApplicationFactory that includes the test setup for our NServiceBus host:

public class TestFactory : WebApplicationFactory<HostApplicationFactoryTests>
{
    public EndpointFixture EndpointFixture { get; }

    public TestFactory() => EndpointFixture = new EndpointFixture();

    protected override IHostBuilder CreateHostBuilder() =>
        Host.CreateDefaultBuilder()
            .UseNServiceBus(ctxt =>
            {
                var endpoint = new EndpointConfiguration("HostApplicationFactoryTests");

                endpoint.ConfigureTestEndpoint();

                return endpoint;
            })
            .ConfigureWebHostDefaults(b => b.Configure(app => {}));

In the above example, we're using a Worker Service that doesn't run a web host. However, the WebApplicationFactory still expects one, so we create an empty web application. The ConfigureTestEndpoint method configures our endpoint to run in a test mode (turn off auditing, retries, etc.)

Finally, in our test class, we add our fixture:

public class HostApplicationFactoryTests 
    : IClassFixture<HostApplicationFactoryTests.TestFactory>
{
    private readonly TestFactory _factory;

    public HostApplicationFactoryTests(TestFactory factory) => _factory = factory;

Now in our test, we execute our "Send" and wait for a message to be handled that we expect:

[Fact]
public async Task Can_send_and_wait()
{
    var firstMessage = new FirstMessage {Message = "Hello World"};

    var session = _factory.Services.GetService<IMessageSession>();

    var result = await _factory
        .EndpointFixture
        .ExecuteAndWaitForHandled<FinalMessage>(() => session.SendLocal(firstMessage));

    result.IncomingMessageContexts.Count.ShouldBe(3);
    result.OutgoingMessageContexts.Count.ShouldBe(3);

    result.ReceivedMessages.ShouldNotBeEmpty();

    var message = result.ReceivedMessages.OfType<FinalMessage>().Single();

    message.Message.ShouldBe(firstMessage.Message);
}

This simple workflow is 3 different message handlers in a row, and kick things off with an initial message. However, we can get more complicated and create a fixture that includes multiple different test hosts:

public class SystemFixture : IDisposable
{
    public WebAppFactory WebAppHost { get; }

    public WebApplicationFactory<Program> WorkerHost { get; }

    public ChildWorkerServiceFactory ChildWorkerHost { get; }

    public EndpointFixture EndpointFixture { get; }

Our integration test can now send an API call via the web host, and it kick off the messages to execute the entire saga until it's complete:

[Fact]
public async Task Should_execute_orchestration_saga()
{
    var client = _fixture.WebAppHost.CreateClient();

    var message = Guid.NewGuid().ToString();

    var response =
        await _fixture.EndpointFixture.ExecuteAndWaitForHandled<SaySomethingResponse>(() =>
            client.GetAsync($"saysomething?message={message}"), 
            TimeSpan.FromSeconds(30));

    var saySomethingResponses = response.ReceivedMessages.OfType<SaySomethingResponse>().ToList();
    saySomethingResponses.Count.ShouldBe(1);
    saySomethingResponses[0].Message.ShouldContain(message);
}

This snippet executes the entire workflow:

It kicks off the test with the initial POST, then waits until the final REPLY to complete the test. Each application runs in a test host, listening to messages, sending messages, making API calls etc. I just need to know what "Done" should be, so I wait for the SaySomethingResponse message to be handled.

The EndpointFixture class also lets me listen for messages sent, in case "done" is some message that gets sent but not handled by these systems.

I find these kinds of integration tests especially useful for process manager/saga scenarios, where I'm pushing over the first domino, and I want to test that the last domino falls successfully. I still expect to have tests all along the way, but I want to have something that tests the entire scenario end-to-end in an environment that runs the entire stack.

In the next post, I'll walk through the code in the integration tests extensions project to show how I used diagnostics events and Reactive Extensions to "know" when to stop.