When a client makes an HTTP request, the client can abort the request, leaving the server processing if it’s not prepared to handle this scenario; wasting its resources that could be used to process other jobs.
In this post, I’ll show how to use Cancellation Tokens to cancel running requests that were aborted by clients.
I’ll not explain how Cancelation Tokens work in this post. I recommend reading this great post by Mitesh Shah that explains how they work in a simple way.
There are two main scenarios why requests are aborted by the client:
When loading a page on the browser, a user may click on the stop button to abort the request, or click the refresh button to reload the page.
When a time-out occurs in the client side. A time-out happens when the time that the client is willing to wait for the response expires. When it happens, the client abandons the request and returns and error. It’s a good practice to have a time-out configured on the client side when making a request through the network, so it doesn’t hang for a long time when the server takes a long time to respond.
First, it’s a waste of resources. Memory and CPU that could be used to process other requests are used to process a request that was already discarded by the client. This waste of resources extends to dependencies, like databases and APIs that the system consumes.
Second, the abandoned request may slow down other requests, competing for shared resources, like database tables.
🚨 Be careful what requests you cancel. It may not be a good idea to abort a request that makes changes to the state of the system, even if the requests are idempotent, as it can make the state inconsistent. It may be better to let the request finish.
Here are good practices when using Cancellation Tokens. (Extracted from Andrew Arnott’s post):
1 - Know when you’ve passed the point of no cancellation. Don’t cancel if you’ve already incurred side-effects that your method isn’t prepared to revert on the way out that would leave you in an inconsistent state. So if you’ve done some work, and have a lot more to do, and the token is cancelled, you must only cancel when and if you can do so leaving objects in a valid state. This may mean that you have to finish the large amount of work, or undo all your previous work (i.e. revert the side-effects), or find a convenient place that you can stop halfway through but in a valid condition, before then throwing OperationCanceledException. In other words, the caller must be able to recover to a known consistent state after cancelling your work, or realize that cancellation was not responded to and that the caller then must decide whether to accept the work, or revert its successful completion on its own.
2 - Propagate your CancellationToken to all the methods you call that accept one, except after the “point of no cancellation” referred to in the previous point. In fact if your method mostly orchestrates calls to other methods that themselves take CancellationTokens, you may find that you don’t personally have to call CancellationToken.ThrowIfCancellationRequested() at all, since the async methods you’re calling will generally do it for you.
3 - Don’t throw OperationCanceledException after you’ve completed the work, just because the token was signaled. Return a successful result and let the caller decide what to do next. The caller can’t assume you’re cancellable at a given point anyway so they have to be prepared for a successful result even upon cancellation.
4 - Input validation can certainly go ahead of cancellation checks (since that helps highlight bugs in the calling code).
5 - Consider not checking the token at all if your work is very quick, or you propagate it to the methods you call. That said, calling CancellationToken.ThrowIfCancellationRequested() is pretty lightweight so don’t think too hard about this one unless you see it on perf traces.
6 - Check CancellationToken.CanBeCanceled when you can do your work more efficiently if you can assume you’ll never be canceled. CanBeCanceled returns false for CancellationToken.None, and in the future possibly for other cases as well.
1 - Add an
CancellationToken object as a parameter in the Controller method. ASP.NET will provide the object through model binding for us.
2 - Pass the
CancellationToken down to all async methods in your code.
3 - The CancellationToken will throw an
TaskCanceledException when the request is aborted. It’s important to log this error to measure how much it is happening.
Refit is a .NET library that facilitates consuming REST APIs. It generates a typed client based on an interface. More details here.
To pass a
CancellationToken to a Refit client as in the example above, just insert a
CancellationToken parameter in the interface:
⚠️ By convention, the
CancellationTokenshould be the last parameter in the method. There is even a static analyzer for this rule (CA1068: CancellationToken parameters must come last). More on static analyzers in this post.
We can manually test if the solution works by inserting a delay in our code and making a request through the browser to the
1 - Insert a 2 minute delay so we have time to stop the request on the client side. It’s required to also pass the
CancellationToken to the
Delay method so it can exit on a request cancellation:
2 - Insert a breakpoint inside the catch
3 - Run the application and open the route URL in the browser (in this example,
4 - Hit the browser’s stop button or press the
We will hit the breakpoint, confirming the solution is working.
- AWS API Gateway has a time-out but it does not report the time-out event to the underlying AWS lambda function. If a time-out occurs at the API Gateway, the users will receive a time-out error but the metrics and logs won’t show those errors, making it seem like no problems are happening.
- Some HTTP Client libraries (for example, Python
net/http) don’t have a default timeout, if the consumer doesn’t set one, it will wait indefinitely for the server response. This may leave the server processing for a long time in case of a bug, for example.
In this post, I explain a complementary solution that mitigates the problems above.