Data Source Providers
Client applications may need to perform operations on the data source when working with the cache. To support this, NCache provides transparent read and write operations on the data source through Read-through, Write-through, and Write-behind caching. This feature enables client applications to read from or write to the data source via the cache when required.
Your provider code will be executed in the same process where your cache is running.
Read-through, Write-through, and Write-behind caching facilitates the following scenarios:
- The user needs to fetch data from the data source in case it is not present in the cache store (Read-through).
- The user needs to update data in the data source synchronously after updating the cache store (Write-through).
- The user needs to update data in the data source asynchronously after updating the cache store (Write-behind).
Advantages of Data Source Providers in NCache
Using this feature in NCache has the following advantages:
- NCache provides a pluggable architecture for Read-through and Write-through caching, so the client application code does not need to contain data source logic.
- The data source may change at any time, so this pluggable architecture can configure different data sources while keeping the application code intact.
- Your application only needs to communicate with NCache and NCache will take care of any backend communication with the data source.
Read-Through Caching
Read-through caching allows NCache to communicate with your data source using a custom provider. It enables NCache to automatically load data behind the get call in case of a cache miss or forcefully via forced Read-through, ensuring that your clustered cache stays synchronized with the data source.
NCache uses your custom Read-through provider to communicate with the data source. In Read-through Caching, NCache will call your provider to load data behind the get call, in case of a cache miss. In clustered caches, where multiple servers are involved, the Read-through provider will be active (initialized) on all cache server nodes, but Read-through operations will only be performed by the node that receives the get operation according to the topology being used.
Forced Read-through
Just like Read-through caching, NCache also provides the option of forced Read-through through which the data is fetched from the data source forcefully, regardless of whether the data is present in the cache or not. This means that data will not be checked in the cache and will be fetched directly from the data source.
Note
NCache provides a performance counter Read-thru/sec for Read-through operations.
Resync mechanism
The Resync mechanism in NCache keeps the data up to date across the cache. There may be a scenario where the user wants to keep the data synced with the data source. In this case, the data will be synced with the data source whenever an item with resync enabled expires. This way the data in the cache remains fresh and operations are performed on updated data. The NCache Read-through provider can be used as a resync provider by specifying ResyncOptions while adding a cache item into the cache. ResyncOptions takes a flag that indicates whether to resync an item on expiration or not. It takes the Read-through provider name as a parameter.
Note
The default Read-through caching provider will be used if the provider name is not specified.
Warning
Do not make calls to the same cache for which Read-through is configured in the provider code. This can cause recursion and can halt the normal working of the cache.
Write-through and Write-behind Caching
Write-through and Write-behind are caching strategies used in NCache to keep cache data consistent with the data source. They provide built-in mechanisms to automatically synchronize a distributed cache with an underlying data source, allowing applications to choose between synchronous consistency or high-performance asynchronous updates based on workload requirements.
NCache supports Write-through caching, which allows the write operations directly on the data source through the cache. This way, you can synchronize your cache and the data source. In Write-through caching, NCache updates the cache store first and then applies that operation to the data source. For example, if a client application updates an entry in the cache, then NCache will also update the data source (if Write-through is enabled).
To use Write-through caching in your application, you need to first implement the IWriteThruProvider interface. NCache will internally use this custom provider to perform write operations on the data source. NCache will call your provider behind write operations (Add, Insert, Remove/Delete) API calls with Write-through.
Currently NCache provides two modes for Write-through caching given below:
- Write-through (Updates data source synchronously)
- Write-behind (Updates data source asynchronously)
Note
NCache provides a performance counter Write-thru/sec for Write-through operations.
What is Write-through Caching
In Write-through caching, an operation is first applied on the cache store and then synchronously updated to the configured data source. Using Write-through, operations will be completed after NCache applies that operation on the data source. You can use Write-through caching if immediate data source updates are critical and you need to update data source as soon as cache is updated.
What is Write-behind caching
In Write-through, due to synchronous operations on the data source, the rate of operations will be same as the rate of user operations on the cache. For applications with high user traffic, the rate of user operations on cache can be very high which can overwhelm your data source. Also, synchronous data source operations may affect response time of a user operation.
To overcome these problems, Write-behind can be used instead of Write-through. In Write-behind, the data source operations are performed asynchronously after NCache performs operations on cache store. After updating the cache store, these operations are queued up and later applied to configured the data sources asynchronously. Thus, using Write-behind will enhance the response time of the cache operations.
NCache provides different configuration settings in Write-behind to control operation flow on the data source. For instance, you can specify the rate at which NCache will apply Write-behind operations on the data source through Throttling.
How Throttling Controls Write-behind Performance
Indicates the number of operations applied on the data source per second. The default value for throttling is 500 ops/sec. You can change this value through the backing source settings in the NCache Management Center.
Note
NCache provides a performance counter Write-behind/sec for Write-behind operations.
Write-behind Modes
NCache allows you to apply Write-behind operations individually or as a batch. A Write-behind queue is maintained for Write-behind operations. All Write-behind operations will be queued in this queue and later applied to the configured data source according to the configured batch or non-batch mode, as explained below.
Non-Batch Mode
By default, non-batch mode will be configured for Write-behind operations. In this mode, operations in Write-behind queue will be applied one by one on data source according to the configured throttling rate. For example, if the throttling rate is 500 operations per second, NCache will apply Write-behind operations one at a time to data source and it will not exceed from 500 operations per second.
Batch Mode
In Batch-mode, a batch/bulk of operations is selected
according to their operation delay. You can configure operation delay for Write-behind
operations, which indicates the time in milliseconds that each operation must
wait in the Write-behind queue before being applied on the data source. By
default, its value is zero.
Batch interval is the configurable interval according to which NCache periodically checks for operation delay timed out operations in Write-behind queue. For example, if operation delay is configured as 1000ms and batch interval as 5 seconds, NCache checks the operations in Write-behind queue after every 5 seconds and selects all operations which have expired operation delays (all operations which are in queue for last 1000 milliseconds).
After selection of operations in bulk, they are applied to the data source according to the configured throttling rate. Let's say a bulk of 1000 operations are selected from the Write-behind queue, which are then applied to the data source in a batch of 500 operations (if throttling rate is 500 ops/sec) as the maximum operation applied to data source per second can't exceed the configured throttling value.
You can specify an operation delay time ranging from seconds to days and months. In this way, you can pause your operations on the data source in a configurable amount of time. NCache also provides performance counters for Write-behind queue, operations count, and current batch operation count. For Write-behind, if batching is enabled, operations that are ready to be executed on data source are dequeued from the Write-behind queue.
Note
The number of operations dequeued in the current batch interval will be displayed by the current batch operation count counter.
Hot Apply Support
NCache supports hot applicable Write-behind settings which allows you to change Write-behind configurations at runtime, without stopping the cache. You can change Write-behind configurable attributes through the NCache Management Center and NCache will incorporate those changes dynamically.
In hot apply support, you can change the Write-behind mode from batch to non-batch and vice versa. For instance, if you have changed the batch mode to non-batch, then NCache will ignore operation delay value and start executing operations individually. Also, you can change the throttling rate at runtime according to your need similarly, operation-delay, batch-interval, failed operations queue limit, and failed operations eviction ratio can also change at runtime.
Warning
You can only increase the value of failed operations queue limit; otherwise, NCache will use its default value for further operations.
Write-behind in Clustered Environment
As a Write-behind queue is maintained for Write-behind operations, a separate dedicated thread monitor executes its operation. The topology level details for Write-behind are mentioned below:
Replicated Topology: Write-behind queue is maintained on all nodes, but Write-behind async processor is present on coordinator node only. It means that all Write-behind operations are performed through this node and replicated to other node queues cluster wide. In this way, if a node is down, then the next coordinator node performs all of the remaining Write-behind operations.
Partition-Replica Topology: Write-behind queue is maintained on each active node and also replicated to its corresponding replicas. Each node is responsible for its Write-behind operation on data source.
Partitioned Topology: Write-behind queue is maintained on each partition and every node is responsible for its Write-behind operations on data source.
Caching Operation Result
NCache provides you with the flexibility to synchronize Write-through operations
in cache on the basis of their operation result. After applying an operation
(Add/Insert) on data source, you can specify operation status on the basis of
which NCache will synchronize the cache store. For example, in case of data
source operation failure, you can decide to remove that data from cache or to
keep it. You can also retry that operation on the data source. For this, you
have to specify Success/Failure/FailureRetry/FailureDontRemove as
OperationResult.Status of OperationResult. This is provided in both
Write-through and Write-behind caching.
Data source operation status and their corresponding actions by NCache are described below:
Success: This means that the data source operation is successful and the item was added to the data source so NCache will keep it in the cache as well.
Failure: This means that data source operation failed and the item could not be added to data source, so NCache will remove it from the cache as well.
FailureDontRemove: This means that the data source operation failed and the item could not be added to the data source, but NCache will keep it in the cache.
FailureRetry: This means that data source operation failed and the item could not be added to the data source, so NCache will keep the item in cache and retry. Retries will be done as Write-behind operations.
Retrying Failed Operations
NCache allows you to retry operations in Write-through/Write-behind in case they are failed on data source. For this purpose, if you enable operation retrying then NCache will retry that operation on data source. In case of Write-through or Write-behind, all retry operations will be requeued to Write-behind queue, which means a Write-through retry operation will be retried asynchronously as a Write-behind operation.
Note
NCache also provides a performance counter for Datasource failed operations/sec.
Write operations performed on data source
returning Failure/FailureRetry/FailureDontRemove as OperationResult.Status of
OperationResult are counted per second by this counter.
NCache allows you to limit the number of failed operations to be retried.
In such a situation, you will mention the failed operation queue limit through the
NCache Management Center and if that limit is exceeded, you can evict failed operations
through a configurable eviction ratio. Here, NCache will evict most retried
operations when retried queue is full. Each operation has a RetryCount property
associated with it, which is incremented on each operation retried on the data
source.
For this, NCache provides a performance counter for
Write-behind failure retry count and Write-behind evictions/sec. The Write-behind failure retry count
shows the number of operations requeued for retry. Data source
write operations returning FailureRetry as Status in OperationResult will be
requeued for retry. Whereas, Write-behind evictions/sec counter displays the
number of retry operations evicted per second.
Updating Cache after Data Source Operation
As stated earlier, in Write-through caching, the operation is first performed on
cache store and then to data source. There may exist scenarios in which after
performing operations on the data source, data becomes modified, e.g. in
case of identity columns, its value may get modified by data source operation.
In such a situation, data may become inconsistent in cache and data source. To
handle this, NCache allows you to specify whether to update data in cache after
data source operations or not. You can enable UpdateInNCache flag to perform
operation (Add/Insert) again on cache store to make it synchronized with the
data source, hence, updating operations in the cache store synchronously
through Write-through or asynchronously through Write-behind caching.
Note
NCache also provides a performance counter Datasource updates/sec that
displays the number of update operations per second in cache after
data source write operations.
To use Read-through or Write-through caching, the IReadThruProvider or the IWriteThruProvider interface needs to be implemented. Runtime parameters can also be provided to the Read-through/Write-through provider while configuring it through the
NCache Management Center or Command Line Tools. These parameters will be passed to the Read-through/Write-through provider on its initialization. These initialization parameters can be used in many ways, e.g., the connection string of a data source can be specified as a parameter, eliminating the need to hardcode it in the provider code.
In NCache, the Read-through/Write-through providers are configured by name, and the provider-specific API can be used by using the provider name. In case of multiple providers, one of the providers will be configured by default. This default provider will be called if a provider name is not mentioned explicitly through the API. Providers other than the default can be used by utilizing provider-specific overloads of the given API.
See Also
Read-through Caching
Write-through Caching
Write-behind Caching
