Alachisoft.com
Download NCache Now!
Contact Us

NCache: Caching Topologies

A caching topology is essentially data storage strategy in a clustered cache. NCache provides a rich set of caching topologies to let you choose the one that suits your environment best. The goal of this is to cater from very small two-server caches to very large cache clusters consisting of hundreds of cache servers. Please note that NCache clustering is not the same as Windows Clustering. NCache forms its own cache-level cluster by using TCP protocol. Below are the main caching topologies provided to you:

  1. Mirrored Cache (2-server active/passive)
  2. Replicated Cache
  3. Partitioned Cache
  4. Partitioned-Replica Cache
  5. Client Cache (a Local Cache connected to a Clustered Cache)
  6. WAN Replication Topologies (Bridge)

Each of these caching topologies is explained in more detail below.

Reference Data versus Transactional Data

Reference data use means you cache something once and read it over and over again. So, there are a lot more reads than writes. On the other hand, transactional data use means that you're updating the data as frequently as you're reading it (or fairly close to it). Originally, cache was considered good only for reference data but distributed caches have proven to be faster and more scalable than databases even for transactional data.

Caching a product catalog where prices change maybe once a day or so is reference use. On the other hand, ASP.NET Session State and JSP Servlet session storage is considered a transactional use.

All topologies are good for reference data but only some topologies are good for transactional data. You need to determine how many updates you'll be doing to figure out which topology is best for you. Below is a list.

  1. Mirrored Cache:
    • Reference data: Good. Max 30000 reads/sec (1k size).
    • Transactional data: Good. Max 20000 writes/sec (1k size).
  2. Replicated Cache:
    • Reference data: Good. 30000 reads/sec per server. Grow linearly by adding servers.
    • Transactional data: Not so good. Max 2500 writes/sec. Drops if you add 3rd server.
  3. Partitioned Cache (no replication):
    • Reference data: Good. 30000 reads/sec per server. Grow linearly by adding servers.
    • Transactional data: Good. 25000 writes/sec per server. Grow linearly by adding servers.
  4. Partition-Replica Cache (with replication):
    • Reference data: Good. 30000 reads/sec per server. Grow linearly by adding servers.
    • Transactional data: Good. 20000 writes/sec per server. Grow linearly by adding servers.

There is some overlap between Mirrored Cache and Partition-Replica Cache for 2-server clusters because both provide similar performance with replication. But, Mirrored Cache accommodates situations where you only have one dedicated cache server and the mirror server is being shared with other apps. But, if you have a need for 3 or more cache servers, then Partition-Replica Cache is the best choice for transactional use.

Mirrored Cache

A Mirrored Cache is a 2-server active/passive cache cluster. All the clients only connect to the active cache server and do their read and write operations against it. For all updates done to the cache (add, insert, and remove) the same updates are also made to the passive server but in the background and as bulk operations. This means that the clients don't have to wait for the updates to be done to the passive server. As soon as the active server is updated, the control returns to the client and then the passive server is updated by a background thread.


Mirrored Cache

This gives Mirrored Cache a significant performance boost over a Replicated Cache of the same size cluster. A Mirrored Cache is almost as fast as a stand-alone Local Cache which has no clustering cost. But, at the same time, a Mirrored Cache provides reliability through replication in case the active cache server goes down.

If the active server ever goes down, the passive server automatically becomes active and all clients automatically connect to this new active server. All of this happens without any interruptions to your application. When we bring the previously active server back up, it joins the cluster and becomes passive since there is now another server that is already active.

Replicated Cache

A Replicated Cache consists of two or more cache servers in a cluster. Each cache server contains the entire cache and any updates to the cache on any server are applied synchronously to all the other servers in the cluster. Replicated Cache ensures that all updates to the cache are made as atomic operations, meaning either all cache servers are updated or none are updated.

The benefit of Replicated Cache is the extremely fast GET performance. Whichever server a client is connected to always has the entire cache. As a result, all GET operations find the data locally on that cache server and this boost the GET speed. However, the cost of an update operation is not very scalable if you want to add servers to a Replicated Cache.

Replicated Cache

A Replicated Cache is ideal for reference data use because you can linearly grow the read capacity by adding more servers to the cluster. But, for transactional data, you can use it in a 2-server cluster configuration but only if your overall load is not very huge. For larger than 2-server clusters, the update performance actually drops.

All updates in a Replicated Cache are made through a sequence-based synchronization algorithm. Whenever a client issues an update request to a cache server, that cache server first contacts the coordinator of the cluster and obtains a unique sequence number. Then, it submits the update operation to all other servers in the cluster along with the sequence number.

Each cache server sequences all update operations based on the sequence number to make sure there is no data integrity problem ever. This means that even if an update operation reaches a cache server first but another operation of an earlier and therefore smaller sequence number has not been performed yet, it waits until all operations with smaller sequence numbers are performed. This ensures consistency of updates across multiple machines.

Partitioned Cache

A Partitioned Cache is intended for larger cache clusters as it is a very scalable caching topology. The cost of a GET or UPDATE operation remains fairly constant regardless of how big your cache cluster is. There are two reasons for it. First of all, the cache partitioning is based on a Hash Map algorithm (similar to a Hashtable). And, a distribution map is created and sent to all the clients that tells the clients which partition has the data or should have the data. This allows the clients to directly go to the cache server that has the data it is looking for.

Partitioned Cache

Secondly, all updates are made to only one server and therefore no sequencing logic is required. Obtaining a sequence adds on extra network round-trip in most cases.

So, not only GET operations are as fast as Replicated Cache, the UPDATE operations are much faster and remain fast regardless of how large the cache cluster gets. This constant cost makes Partitioned Cache a highly scalable topology.

However, please note that there is no replication in Partitioned Cache. So, if any cache server goes down, you lose that much cache. This may be okay in many object caching situations but is not okay when you're using the cache as your main data repository without the data existing in any master data source. A good example of this is ASP.NET Session State storage or JSP Servlet session storage in the cache.

Partitioned-Replica Cache

Partitioned-Replica Cache is a combination of Partitioned Cache and Replicated Cache. It gives you best of both worlds. You get reliability through replication and scalability through partitioning. Instead of replicating the cache over and over again if you have more than 2 servers in the cluster, you only replicate the cache once (meaning only two copies of the cache exist) regardless of how big the cache cluster is. This allows you to scale out through partitioning.

Partitioned-Replica uses the same Hash Map algorithm for data distribution as Partitioned Cache. However, each partition may be replicated to one other server in the cluster and kept as a "passive Replica". Each server contains one partition and one replica of another server's partition. The replica is not directly accessed by any clients as long as it is in the "passive" mode. Only the active Partition accesses it. But, if the active Partition ever goes down (perhaps because that server has gone down), then the passive replica becomes an active partition and starts receiving client requests. Then, this replica is made available as a replacement to its partition.

Partitioned-Replica Cache

Partitioned-Replica provides both sync and async replication. Async replication is the default option. Async replication is really fast because it does not require the user to wait for the replica to be updated as well.

In async replication, all updates to the replica are queued up on the active partition server and a background thread does bulk-updates to the replica. Although, this is super fast but there is still a possibility that if active partition server crashes abruptly (instead of being broken down intentionally), you will lose a small amount of data that hasn't yet made it to the replica. Now, this is okay for most applications but in some situations the data is too sensitive (perhaps itís financial data) to be lost.

For those situations, a sync-replication is provided in Partition-Replica Cache. In this, both the active partition and the replica are updated as an atomic operation. Due to this, the performance is not as fast as the async-replication but it is still much faster and more scalable than Replicated Cache.

It is faster than Replicated Cache because sequence-based synchronization is not needed for updating the replica since the replica is passive and only the active partition is updating it.

And, unlike Replicated Cache where update performance and capacity actually drops as you grow the cache cluster, the same is not true here. This is because there are still only two copies of the data regardless of how big the cache cluster becomes so adding more servers increases update capacity linearly.

Client Cache

A Client Cache is local to your web server or app server and lets you cache frequently used data even closer to your application than the clustered cache. So, in essence, a Client Cache is a cache-on-top-of-a-cache and gives your application further boost in performance and scalability. A Client Cache works with any of the clustered caching topologies (Mirrored, Partitioned, and Partition-Replica Cache).

Client Cache is good for reference data only (so do not use it for ASP.NET Session State or JSP Servlet Session storage). This is because the updates are now being made to both the Client Cache and the clustered cache and are therefore slightly slower than if you didn't even have any Client Cache. But, the reads are much faster than going even to the Clustered Cache (especially if you use InProc setting on the Client Cache).

Client Cache

A Client Cache is basically a stand-alone local cache on your web server or app server but with a difference that it is also connected to the clustered cache and stays synchronized with it. This synchronization means that whatever is in the Client Cache, if the same data is updated in the clustered cache by another application or application instance, then the cluster cache notifies the Client Cache to update itself.

A Client Cache is not part of the cache cluster. So, there is a very small period of time (a few milliseconds) after the clustered cache has been updated when the Client Cache is notified to update itself. This means that you could have situations where data in the Client Cache is not consistent with the clustered cache. To handle this, Client Cache provides two types of synchronizations with the clustered cache:

  1. Optimistic Synchronization: In this mode, the application takes whatever is in the Client Cache and there is a slight possibility that this data may be old because the clustered cache has been updated but the event notification hasn't arrived in the Client Cache yet.
  2. Pessimistic Synchronization: In this mode, the application does not trust the Client Cache and always issues a "GetIfNewerVersion()" call to the clustered cache. This call returns "null" if no new version of the cached item is available in the clustered cache but if data has changed in the clustered cache then it returns it. The performance is of course not as fast as optimistic synchronization but still faster than not having any Client Cache.

WAN Replication Topologies (Bridge)

Read more details here