Hangfire

Hangfire is a robust background job framework for .NET that allows applications to execute logic outside the main web request pipeline. Since Hangfire itself does not store state, it requires a persistent storage backend to manage jobs, queues, execution history, and server coordination.

Integrating NCache with Hangfire allows NCache to serve as the shared, distributed storage engine for managing Hangfire’s job data and state. By replacing traditional SQL or Redis backends with NCache, your background processing pipeline benefits from an in-memory, highly scalable, and distributed architecture where all Hangfire components coordinate through a single shared cache.

Important

Hangfire is supported in NCache 5.3.8 and onwards in NCache and 5.3.6.2 and onwards in NCache OpenSource.

Decoupled Clients and Servers

All Hangfire instances interact exclusively through a shared NCache cluster, eliminating the need for direct client-to-server communication.

  • Hangfire Client (Web App/API): The application submits background jobs to Hangfire, which uses the NCache storage provider to store job payloads, arguments, and parameters in NCache, create execution queue entries, and trigger Pub/Sub queue wake-up signals. It does not execute the background tasks directly.

  • Hangfire Server (Worker Background Process): The worker receives queue wake-up signals via NCache Pub/Sub notifications or polls the queues, retrieves jobs from NCache (lock-free via native distributed queues in Enterprise, or under a distributed lock in Open Source), executes the workload, and persists state back into the cache.

While both editions decouple clients and servers through NCache, their internal storage architecture differs:

Feature Enterprise Open Source
Data Engine Uses native distributed data types (IDistributedQueue, DistributedDictionary, DistributedHashSet, DistributedCounter). Uses plain cache entries with custom head/tail pointer logic.
Execution Path Locking Lock-free hot path. Native distributed queues guarantee concurrency safety without application-level locking. Lock-guarded hot path. Every enqueue, dequeue, and counter update is guarded by NCacheLockGuard and local gates.
Pub/Sub Subscriptions Durable exclusive topics. Notifications persist during worker restarts up to DurableSubscriptionExpiry. Non-durable topics. Workers offline during a broadcast rely on periodic queue sweeps.
Crash Requeue Behavior FetchedJobsWatcher requeues abandoned jobs at the tail of the queue. FetchedJobsWatcher requeues abandoned jobs at the front of the queue.
Dashboard Metrics Automatically registers 5 NCache metric boxes on the dashboard overview page. Standard Hangfire metrics only.

How It Works: The Request Flow

When you enqueue a background task, NCache manages the job from creation to execution across your servers through the following steps:

  1. Job Registration: Your application creates a job through Hangfire, which uses the NCache storage provider to save its details, arguments, and parameters in NCache with an initial expiration.

  2. Queue Insertion: The job ID is appended to the designated queue entry in NCache under a distributed write transaction.

  3. Queue Notification: If Pub/Sub is enabled, NCache sends a queue wake-up signal to waiting workers when new work is available.

  4. Locked Dequeue & Execution: An available Hangfire worker retrieves the job from NCache (using native lock-free distributed queue operations in Enterprise, or distributed locks in Open Source), marks it as in progress, and executes the workload.

  5. State Persistence & Batching: Once finished, terminal state records (Succeeded/Failed) and execution counters are written back to NCache (directly via native types in Enterprise, or grouped in periodic flush batches in Open Source).

Supported Job Types

The NCache storage provider natively manages the backing data structures for standard Hangfire job patterns:

  • Fire-and-Forget: Added directly to the queue in NCache so an available worker can run it immediately.

  • Delayed/Scheduled: Stored with a future execution time in NCache; once that time arrives, Hangfire automatically moves the job into the active queue.

  • Recurring: Job definitions and schedule metadata are stored as NCache hash entries and sets, which Hangfire evaluates periodically.

  • Continuations: Stored in a waiting state in NCache and only queued for execution after their parent job finishes successfully.

Reliability and Crash Recovery

The NCache storage provider includes built-in background safety mechanisms for worker failures and missed queue notifications:

  • Crashed Worker Recovery (FetchedJobsWatcher): While a job is being processed, its FetchedAt timestamp is periodically renewed at approximately one-third of the configured InvisibilityTimeout. If the worker or process crashes, the renewal stops. Once the processing entry becomes stale, FetchedJobsWatcher detects it and requeues the job (placed at the tail of the queue in Enterprise, or at the front in Open Source). Because recovery is timeout-based, jobs should be designed to tolerate possible duplicate execution. Recovery occurs when the timeout has elapsed and the watcher performs its next scan; it does not necessarily occur at the exact moment the timeout expires.

  • Backlog Signal Recovery (QueueBacklogWatcher): If a real-time queue notification is missed, NCache periodically checks for waiting jobs every 10 seconds and sends queue wake-up signals to workers, ensuring that queued tasks continue to be processed.

In This Section

NCache as Hangfire Storage Provider
Learn how to configure and use NCache as an in-memory distributed storage provider for Hangfire in ASP.NET Core applications.