Introduction

When a client makes an HTTP request, the client can abort the request, leaving the server processing if it’s not prepared to handle this scenario; wasting its resources that could be used to process other jobs.

In this post, I’ll show how to use Cancellation Tokens to cancel running requests that were aborted by clients.

Cancelation Tokens

I’ll not explain how Cancelation Tokens work in this post. I recommend reading this great post by Mitesh Shah that explains how they work in a simple way.

Aborted requests

There are two main scenarios why requests are aborted by the client:

• When loading a page on the browser, a user may click on the stop button to abort the request, or click the refresh button to reload the page.

• When a time-out occurs in the client side. A time-out happens when the time that the client is willing to wait for the response expires. When it happens, the client abandons the request and returns and error. It’s a good practice to have a time-out configured on the client side when making a request through the network, so it doesn’t hang for a long time when the server takes a long time to respond.

Why stop processing abandoned requests on the server?

First, it’s a waste of resources. Memory and CPU that could be used to process other requests are used to process a request that was already discarded by the client. This waste of resources extends to dependencies, like databases and APIs that the system consumes.

Second, the abandoned request may slow down other requests, competing for shared resources, like database tables.

🚨 Be careful what requests you cancel. It may not be a good idea to abort a request that makes changes to the state of the system, even if the requests are idempotent, as it can make the state inconsistent. It may be better to let the request finish.

Here are good practices when using Cancellation Tokens. (Extracted from Andrew Arnott’s post):

1 - Know when you’ve passed the point of no cancellation. Don’t cancel if you’ve already incurred side-effects that your method isn’t prepared to revert on the way out that would leave you in an inconsistent state. So if you’ve done some work, and have a lot more to do, and the token is cancelled, you must only cancel when and if you can do so leaving objects in a valid state. This may mean that you have to finish the large amount of work, or undo all your previous work (i.e. revert the side-effects), or find a convenient place that you can stop halfway through but in a valid condition, before then throwing OperationCanceledException. In other words, the caller must be able to recover to a known consistent state after cancelling your work, or realize that cancellation was not responded to and that the caller then must decide whether to accept the work, or revert its successful completion on its own.

2 - Propagate your CancellationToken to all the methods you call that accept one, except after the “point of no cancellation” referred to in the previous point. In fact if your method mostly orchestrates calls to other methods that themselves take CancellationTokens, you may find that you don’t personally have to call CancellationToken.ThrowIfCancellationRequested() at all, since the async methods you’re calling will generally do it for you.

3 - Don’t throw OperationCanceledException after you’ve completed the work, just because the token was signaled. Return a successful result and let the caller decide what to do next. The caller can’t assume you’re cancellable at a given point anyway so they have to be prepared for a successful result even upon cancellation.

4 - Input validation can certainly go ahead of cancellation checks (since that helps highlight bugs in the calling code).

5 - Consider not checking the token at all if your work is very quick, or you propagate it to the methods you call. That said, calling CancellationToken.ThrowIfCancellationRequested() is pretty lightweight so don’t think too hard about this one unless you see it on perf traces.

6 - Check CancellationToken.CanBeCanceled when you can do your work more efficiently if you can assume you’ll never be canceled. CanBeCanceled returns false for CancellationToken.None, and in the future possibly for other cases as well.

Canceling the requests on ASP.NET Core

1 - Add an CancellationToken object as a parameter in the Controller method. ASP.NET will provide the object through model binding for us.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

[ApiController]
[Route("[controller]")]
public class DogImageController : ControllerBase
{
    private readonly IDogImageUseCase _dogImageUseCase;
    private readonly ILogger<DogImageController> _logger;

    public DogImageController(IDogImageUseCase dogImageUseCase, ILogger<DogImageController> logger)
    {
        _dogImageUseCase = dogImageUseCase;
        _logger = logger;
    }

    [HttpGet]
    public async Task<ActionResult<string>> GetAsync(CancellationToken cancellationToken)
    {
        try
        {
            return await _dogImageUseCase.GetRandomDogImage(cancellationToken);
        }
        catch(TaskCanceledException ex)
        {
            _logger.LogError(ex, ex.Message);

            return StatusCode(StatusCodes.Status500InternalServerError, "Request cancelled");
        }
    }
}

2 - Pass the CancellationToken down to all async methods in your code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

public class DogImageUseCase: IDogImageUseCase
{
    private readonly IDogApi _dogApi;

    public DogImageUseCase(IDogApi dogApi)
    {
        _dogApi = dogApi;
    }

    public async Task<string> GetRandomDogImage(CancellationToken cancellationToken)
    {
        var dog = await _dogApi.GetRandomDog(cancellationToken);

        return dog.message;
    }
}

3 - The CancellationToken will throw an TaskCanceledException when the request is aborted. It’s important to log this error to measure how much it is happening.

CancellationToken with Refit

Refit is a .NET library that facilitates consuming REST APIs. It generates a typed client based on an interface. More details here.

To pass a CancellationToken to a Refit client as in the example above, just insert a CancellationToken parameter in the interface:

1
2
3
4
5

public interface IDogApi
{
    [Get("/breeds/image/random")]
    Task<Dog> GetRandomDog(CancellationToken cancellationToken);
}

⚠️ By convention, the CancellationToken should be the last parameter in the method. There is even a static analyzer for this rule (CA1068: CancellationToken parameters must come last). More on static analyzers in this post.

Testing the solution

We can manually test if the solution works by inserting a delay in our code and making a request through the browser to the /DogImage route.

1 - Insert a 2 minute delay so we have time to stop the request on the client side. It’s required to also pass the CancellationToken to the Delay method so it can exit on a request cancellation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

public class DogImageUseCase: IDogImageUseCase
{
    private readonly IDogApi _dogApi;

    public DogImageUseCase(IDogApi dogApi)
    {
        _dogApi = dogApi;
    }

    public async Task<string> GetRandomDogImage(CancellationToken cancellationToken)
    {
        var dog = await _dogApi.GetRandomDog(cancellationToken);

        await Task.Delay(TimeSpan.FromMinutes(2), cancellationToken);

        return dog.message;
    }
}

2 - Insert a breakpoint inside the catch TaskCanceledException block;

3 - Run the application and open the route URL in the browser (in this example, /DogImage);

4 - Hit the browser’s stop button or press the esc key.

We will hit the breakpoint, confirming the solution is working.

Problems with this solution

• AWS API Gateway has a time-out but it does not report the time-out event to the underlying AWS lambda function. If a time-out occurs at the API Gateway, the users will receive a time-out error but the metrics and logs won’t show those errors, making it seem like no problems are happening.
• Some HTTP Client libraries (for example, Python requests and Go net/http) don’t have a default timeout, if the consumer doesn’t set one, it will wait indefinitely for the server response. This may leave the server processing for a long time in case of a bug, for example.

In this post, I explain a complementary solution that mitigates the problems above.

References and links

• Microsoft’s CancellationToken Documentation
• ASP.NET Core’s Model Binding
• A Deep Dive into C#’s CancellationToken