Redis Cluster is finally supported. Also improved connection resilience using custom connection loop, upgraded StackExchange.Redis to the latest version and fixed some nasty bugs. So I’d recommend to upgrade, especially if you are using Redis in cloud environments.

Redis Cluster

Although StackExchange.Redis works with Redis Cluster out of the box, we couldn’t just pass cluster endpoints to the constructor of the RedisStorage class and relax, expecting all is working fine. Hangfire uses distributed locks to coordinate the work between multiple machines, and proper implementation of distributed locks using Redis requires us to use the RedLock algorithm. Because replication may fail in very unpleasant places, due to its asynchronous nature in Redis.

But RedLock requires to have minimum 3 Redis instances, detached from the cluster, just to ensure distributed locks are working correctly in corner cases. Since every additional unit of infrastructure greatly increases the complexity of the system in total, I was investigating other ways to keep things simple. And recently I’ve realized that since distributed locks are only protecting writes to job storage, it’s enough to use the WATCH command for acquired locks before executing a transaction. So simple, so good, just specify the hash slot using curly braces in prefix:

GlobalConfiguration.Configuration.UseRedisStorage(
    "redis1,redis2,redis3",
    new RedisStorageOptions { Prefix = "{hangfire}:" });

Connection Resilience

I’ve found out that under some rare circumstances (manual failover also leads to this problem), connection to a Redis instance isn’t restored automatically, until we re-create a multiplexer manually. So I’ve implemented a custom reconnect policy to handle these cases, without the need to restart a Hangfire Server instance. This feature is particularly useful, if you are using cloud provider to host your Redis instances.

Release Notes

• Added – Redis Cluster is fully supported now without requiring RedLock algorithm and additional masters.
• Added – Provide custom certificate selection/validation callbacks via RedisStorageOptions class.
• Changed – Custom reconnect logic to solve StackExchange.Redis rare connectivity problems in the cloud.
• Changed – StackExchange.Redis updated to 1.2.1, a lot of problems fixed there.
• Changed – A new state entry is added for background jobs re-queued due to elapsed invisibility timeout.
• Changed – Improve distributed lock integrity by using WATCH conditions before running transactions.
• Changed – Decrease expiration time for distributed lock to 1 minute, will reduce delays caused by abandoned locks.
• Changed – Default ConnectTimeout and ResponseTimeout increased to 15s and 5 minutes accordingly.
• Changed – Disable TieBreaker and ConfigurationChannel – multiple standalone masters aren’t supported anyway.
• Changed – Remove dependency on the Microsoft.NETCore.Portable.Compatibility package.
• Changed – Assembly is now marked as CLS Compliant.
• Fixed – Connection to Redis can’t be restored under rare circumstances.
• Fixed – Jobs with missing type information don’t cause infinite retries and problems with processing.
• Fixed – Idle workers don’t pick up a forcibly re-queued job until a new one is added.
• Fixed – Forward clock change doesn’t cause non-initialized jobs to be expired before processing.
• Deprecated – RedisStorageOptions.AllowMultipleEndPointsWithoutRedLock is now obsolete, just remove it.
• Deprecated – RedisStorageOptions.SubscriptionIntegrityTimeout is now obsolete, just remove it.