This release comes with great feature for long-running jobs: cancellation tokens. They are like
CancellationToken class, but for background jobs.
Previously, you was not able to stop a running job by using
Delete methods of the
BackgroundJob class, or by clicking a button in Hangfire Monitor. Its state was changed, but the job is still running. And running… And running… Nobody waits for it, why it is not stopped yet?
Job cancellation tokens provide
ThrowIfCancellationRequested method that throws
OperationCanceledException if a job was canceled due to:
Shutdown request – executed on
BackgroundJobServer.Stopmethod invocation. Performed automatically when ASP.NET application is shutting down.
State transition – the state of the job has been changed, and it is not in the
BackgroundJob.Requeueas well as all Monitor interface buttons lead to the job cancellation (in case it is running).
To use cancellation tokens, you need to add just one parameter of type
IJobCancellationToken to a target method:
Then, create a background job:
To be able to test target methods, or to add the support of cancellation token to your old jobs, you can use the
This release solved another problem. On some job queues (Redis, SQL Server, but not MSMQ) it is not possible to apply transaction just to fetch a job from a queue, hide it from another workers, and delete it on successful processing or place it back to a queue on failure or process termination.
To mimic this behavior, Hangfire uses atomic get/set commands:
output clause in SQL Server and
BRPOPLPUSH in Redis. That is why other workers don’t see just fetched job. But in case of process termination, all jobs will remain in invisible state. To fight with this, there is a component on a server, who seeks invisible jobs and decides what to do with them.
Unfortunately, there is no way to determine whether a job was aborted by a process termination, or it is still working. To separate these cases, the component checks the fetching time of a job and compares it with current time. If the result is greater than InvisibilityTimeout, then we should trait the job as hanged, and return it to its queue. InvisibilityTimeout defaults to 30 minutes to be sure that the job processing was aborted.
When ASP.NET issues shutdown request and gives background jobs 30 seconds to die, some of them may be aborted by
ThreadAbortException. In previous releases this lead to the fact that background job may stay in Invisible state on regular shutdown, and this sometimes increase the latency of job processing.
In this release Hangfire make an attempt to place aborted job back to the start of its queue on this exception, and it will be started again immediately after application restart on success, and after InvisibilityTimeout on failure.
When the feature of re-queueing jobs on
ThreadAbortException plays together with job cancellation tokens, it means that you greatly decrease the latency of background job processing, because the probability of using
InvisibilityTimeout is greatly decreased.
Use job cancellation tokens where possible to ensure that your jobs are shutting down gracefully, and you greatly decrease the probability of high latencies in your background job processing.
- Added – Cancellation token for job methods that throws on server shutdown and job aborts.
- Added – Place interrupted job back to its queue if possible.
- Fixed – Can not delete jobs when method or class was removed.
- Fixed – NullReferenceException in Monitor.
- Fixed – SqlException when changing state of a job with absent target method.