Time-out requests in ASP.NET Core with cancellation tokens

Introduction

When ASP.NET Core is running in an AWS Lambda and receiving requests through an AWS API Gateway, the application is not notified of an API Gateway time-out and keeps processing the request, completing it eventually. This will leave metrics and logs of a successful request when the client received a time-out error.

In this post, I’ll show how to solve this problem with cancellation tokens and time-outs.

The solution

The idea is to time-out the request in the ASP.NET application before the API Gateway time-out. This way we can collect logs and metrics of the requests that timed out and have a realistic view of the errors the users are seeing.

In this previous post, I’ve explained how to cancel requests aborted by the HTTP client using the CancellationToken in the controllers methods.

The default ASP.NET’s model binding for the CancellationToken injects the RequestAborted property of the HttpContext object in the controller methods. The RequestAborted property holds a CancellationToken that is triggered when the client abandons the request.

Let’s create a middleware to apply the timeout to ASP.NET Core’s Cancellation Token.

First, we’ll create a new CancellationTokenSource and set a time-out to it.

Then, we’ll use the CancellationToken.CreateLinkedTokenSource method to link the HttpContext.RequestAborted CancellationToken to the new CancellationToken we created with a time-out.

Lastly, we override the HttpContext.RequestAborted Cancellation Token with the token returned by the CreateLinkedTokenSource.

⚠️ Remember to always dispose of the CancellationTokenSource objects to avoid memory leaks. Use the using keyword or dispose of them in the HttpContext.Response.OnCompleted delegate.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
public class TimeoutCancellationMiddleware
{
    private readonly RequestDelegate _next;
    private readonly TimeSpan _timeout;

    public TimeoutCancellationMiddleware(RequestDelegate next, TimeoutCancellationMiddlewareOptions options)
    {
        _next = next;
        _timeout = options.Timeout;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        //Create a new CancellationTokenSource and set a time-out
        using var timeoutCancellationTokenSource = new CancellationTokenSource();
        timeoutCancellationTokenSource.CancelAfter(_timeout);

        //Create a new CancellationTokenSource linking the timeoutCancellationToken and ASP.NET's RequestAborted CancellationToken
        using var combinedCancellationTokenSource =
            CancellationTokenSource
                .CreateLinkedTokenSource(timeoutCancellationTokenSource.Token, context.RequestAborted);

        //Override the RequestAborted CancellationToken with our combined CancellationToken
        context.RequestAborted = combinedCancellationTokenSource.Token;

        await _next(context);
    }
}

public class TimeoutCancellationMiddlewareOptions
{
    public TimeSpan Timeout { get; set; }

    public TimeoutCancellationMiddlewareOptions(TimeSpan timeout)
    {
        Timeout = timeout;
    }
}

We can also create an extension method to make it easier to configure the middleware:

1
2
3
4
5
6
7
8
9
public static class TimeoutCancellationMiddlewareExtensions
{
    public static IApplicationBuilder UseTimeoutCancellationToken(
        this IApplicationBuilder builder, TimeSpan timeout)
    {
        return builder.UseMiddleware<TimeoutCancellationMiddleware>(
            new TimeoutCancellationMiddlewareOptions(timeout));
    }
}

Then, add it to the middleware pipeline in our application:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
public class Program
{
    public static void Main(string[] args)
    {
        var builder = WebApplication.CreateBuilder(args);

        builder.Services.AddControllers();

        var app = builder.Build();

        ...

        //Configure a request time-out of 10 seconds
        app.UseTimeoutCancellationToken(TimeSpan.FromSeconds(10));

        ...

        app.Run();
    }
}

Now, we have to use the CancellationToken injected in the controllers’ methods and pass it down to all async methods.

DogImageController:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
[ApiController]
[Route("[controller]")]
public class DogImageController : ControllerBase
{
    private readonly IDogApi _dogApi;
    private readonly ILogger<DogImageController> _logger;

    public DogImageController(IDogApi dogApi, ILogger<DogImageController> logger)
    {
        _dogApi = dogApi;
        _logger = logger;
    }

    [HttpGet]
    public async Task<ActionResult<string>> GetAsync(CancellationToken cancellationToken)
    {
        try
        {
            var dog = await _dogApi.GetRandomDog(cancellationToken);

            return dog.message;
        }
        catch(TaskCanceledException)
        {
            _logger.LogError(ex, ex.Message);

            return StatusCode(StatusCodes.Status500InternalServerError, "Request timed out or canceled");
        }
    }
}

🚨 Do not catch TaskCanceledException anywhere except in the controller, or else your API can return an empty 200 / OK result.

DogImageUseCase:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
public class DogImageUseCase: IDogImageUseCase
{
    private readonly IDogApi _dogApi;

    public DogImageUseCase(IDogApi dogApi)
    {
        _dogApi = dogApi;
    }

    public async Task<string> GetRandomDogImage(CancellationToken cancellationToken)
    {
        var dog = await _dogApi.GetRandomDog(cancellationToken);

        return dog.message;
    }
}

Testing the solution

To test if it works, let’s force a delay in the DogImageUseCase class:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
public class DogImageUseCase: IDogImageUseCase
{
    private readonly IDogApi _dogApi;

    public DogImageUseCase(IDogApi dogApi)
    {
        _dogApi = dogApi;
    }

    public async Task<string> GetRandomDogImage(CancellationToken cancellationToken)
    {
        var dog = await _dogApi.GetRandomDog(cancellationToken);

        await Task.Delay(TimeSpan.FromMinutes(1), cancellationToken);

        return dog.message;
    }
}

Creating an integration test for the time-out scenario

In this previous post, I explained how to use WireMock.Net to mock API dependencies in integration tests.

Now, we’ll use WireMock.Net to create a mock that responds with a delay. This way, we can validate that our application is stopping the processing of the request when the time-out occurs.

First, we override the HttpClientTimeoutSeconds configuration value to a time-out bigger than our request time-out. This configuration is used to set the Timeout property of the HttpClient. If the HttpClient time-out is smaller than our request time-out, it will abort the request to the dependencies before the application times out.

Then, we use WireMock’s AddGlobalProcessingDelay to insert a delay in the API mock and force our application to time-out before the response.

Lastly, we assert that our application returned the status code 500 with the message Request timed out or canceled.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
public class DogImageTests: IClassFixture<WebApplicationFactory<Program>>
{
    private readonly WebApplicationFactory<Program> _factory;

    public DogImageTests(WebApplicationFactory<Program> factory)
    {
        _factory = factory;
    }

    ...

    [Fact]
    public async Task Timeout_Returns_500WithMessage()
    {
        //Arrange
        var wireMockSvr = WireMockServer.Start();

        var factory = _factory
            .WithWebHostBuilder(builder =>
            {
                builder.UseSetting("DogApiUrl", wireMockSvr.Url);

                //Set the HttpClient timeout to 30, so it doesn't trigger before our request timeout that is 10 seconds
                builder.UseSetting("HttpClientTimeoutSeconds", "30");

                //Set the Request time-out to 5 seconds for the test to run faster
                builder.UseSetting("RequestTimeoutSeconds", "5");  
            });

        var httpClient = factory.CreateClient();

        Fixture fixture = new Fixture();

        var responseObj = fixture.Create<Dog>();
        var responseObjJson = JsonSerializer.Serialize(responseObj);

        wireMockSvr
            .Given(Request.Create()
                .WithPath("/breeds/image/random")
                .UsingGet())
            .RespondWith(Response.Create()
                .WithBody(responseObjJson)
                .WithHeader("Content-Type", "application/json")
                .WithStatusCode(HttpStatusCode.OK));

        //Add a delay to the response to cause a request timeout in the /DogImage endpoint
        wireMockSvr.AddGlobalProcessingDelay(TimeSpan.FromSeconds(15));

        //Act
        var apiHttpResponse = await httpClient.GetAsync("/DogImage");

        //Assert
        apiHttpResponse.StatusCode.Should().Be(HttpStatusCode.InternalServerError);

        var responseMessage = await apiHttpResponse.Content.ReadAsStringAsync();

        responseMessage.Should().BeEquivalentTo("Request timed out or canceled");

        wireMockSvr.Stop();
    }
}

Complete code

Check it on the GitHub Repository.