Caching Topologies
NCache provides a rich set of caching topologies to let you choose the one that suits your environment best. The goal of this is to cater from very small single-node caches to very large cache clusters consisting of hundreds of cache servers. Below are the main caching topologies provided to you:
- Local Cache (stand-alone)
- Mirrored Cache (2-node active/passive with replication)
- Replicated Cache
- Partitioned Cache
- Partitioned-Replica Cache
- Client Cache (a Local Cache connected to a Clustered Cache)
- Bridge Topology
Each of these caching topologies are explained in more detail below.
Local Cache
Local Cache is a stand-alone cache and is good for single server environments. You can either access it InProc or OutProc. An OutProc local cache lives in the Alachisoft.NCache.Service.exe process and is not affected if your application process recycles (e.g. ASP.NET worker processes).
Figure 1: Local cache with local and remote clients
Please note that a Local Cache also accepts remote clients just like a clustered cache. However, the cache itself is stand-alone and lives only on one machine. A Local Cache is just like an ASP.NET Cache, a Caching Application Block (CAB) Cache, or NHibernate cache except that it allows OutProc and remote client access.
If you have a .NET application using one of the standard InProc caches (e.g. ASP.NET Cache, Caching Application Block, or NHibernate Level-1 Cache), and you want the cache to be hosted in an separate process, you can configure a Local Cache with OutProc option.
Mirrored Cache
A Mirrored Cache is a 2-node cache cluster. In this topology, one cache server is active and the second one passive. All the clients only connect to the active cache server and do their read and write operations again it. For all updates done to the cache (add, insert, and remove) the same updates are also made to the passive server but in the background. This means that the clients don't have to wait for the updates to be done to the passive server. As soon as the active server is updated, the control returns to the client and then the passive server is updated by a background thread.
Figure 2: Mirrored Cache with remote clients
This gives Mirrored Cache a significant performance boost over a Replicated Cache or Partitioned-Replica Cache of the same size cluster. A Mirrored Cache is almost as fast as a Local Cache which has no clustering cost. At the same time, a Mirrored Cache provides reliability in case the active cache server goes down.
If the active server goes down, the passive server automatically becomes active and all clients also automatically connect to this new active server. In this situation, you can either bring the previously active server back up as a passive server now or add another server to the cluster as a passive server.
Replicated Cache
A Replicated Cache consists of two or more cache servers in a cluster. Each cache server contains the entire cache and any updates to the cache on any server are applied synchronously to all the other servers in the cluster. Replicated Cache ensures that all updates to the cache are made as atomic operations, meaning either all cache servers are updated or none are updated.
The benefit of Replicated Cache is the extremely fast GET performance. Whichever server a client is connected to always has the entire cache. As a result, all GET operations find the data locally on that cache server and this boost the GET speed. However, the cost of an update operation is not very scalable if you want to add servers to a Replicated Cache.
Figure 3: Replicated Cache
A Replicated Cache is good for 2-3 server cache clusters for a transactional application where 30% or more of your operations are writes. However, if your application is read-intensive and infrequently does updates, then even larger Replicated Cache clusters work just fine. The only issue with larger Replicated Cache clusters is the frequency of the update operations (Add, Insert, and Remove).
All updates in a Replicated Cache are made through a sequence-based logic. Whenever a client issues an update request to a cache server, that cache server first contacts the coordinator of the cluster and obtains a unique sequence number. Then, it submits the update operation to all other servers in the cluster along with the sequence number.
Each cache server sequences all update operations based on the sequence number to make sure there is no data integrity problem ever. This means that even if an update operation reaches a cache server first but another operation of an earlier and therefore smaller sequence number has not been performed yet, it waits until all operations with smaller sequence numbers are performed. This ensures consistency of updates across multiple machines.
Partitioned Cache
A Partitioned Cache is intended for larger cache clusters as it is a very scalable caching topology. The cost of a GET or UPDATE operation remains fairly constant regardless of how big your cache cluster is. There are two reasons for it. First of all, the cache partitioning is based on a Hash Map algorithm (similar to a Hashtable) so if the operation is not local to the given cache server, the cache server knows exactly which other cache server is involved. So, there is only one extra network trip and the cache server does not have to inquire from all other cache servers about the location of the data involved in this operation. So, in essence, a GET or UPDATE operation is either local to a cache server or one hop away in the cluster.
Figure 4: Partitioned Cache
Secondly, all updates are made to only one server and therefore no sequencing logic is required. Obtaining a sequence adds on extra network round-trip in most cases.
So, although a GET may not be as fast as Replicated Cache because it does not always find the data local to the given cache server, its cost is fairly constant even when the cluster size grows. This constant cost makes Partitioned Cache a highly scalable topology.
However, please note that there is no replication in Partitioned Cache. So, if any cache server goes down, you lose that much cache. This may be okay in many object caching situations but is not okay when you're using the cache as your main data repository without the data existing in any master data source. A good example of this is ASP.NET Session State storage in the cache.
NCache 3.4 has added an important performance and scalability enhancement to Partitioned and Partitioned-Replica Cache. And, this is to allow (as an option) the clients to connect to all the cache servers in the cluster. Additionally, each client now receives a copy of the data distribution map (the one that cache servers use to figure out which data resides on which server). This allows the clients to directly talk to the appropriate server based on what data the client is dealing with. Below is a diagram showing this new configuration.
Figure 5: Client connection optimization
Previously, each client only connected to one cache server and if the data it was requesting didn't reside on that cache server, that cache server would fetch it from another cache server in the client. Although, it was totally transparent to the client but still it involved an extra network trip. That option is still available in NCache 3.4 but turned off by default.
Partitioned-Replica Cache
Partitioned-Replica Cache is a combination of Partitioned Cache and Replicated Cache. It gives you best of both worlds. You get reliability through replication and scalability through partitioning. Instead of replicating the cache over and over again depending on how many servers you have in the cluster, you only replicate the cache once (meaning only two copies of the cache exist regardless of how big the cache cluster is). This allows you to scale up through partitioning.
Partitioned-Replica uses the same Hash Map algorithm for data distribution as Partitioned Cache. However, each partition may be replicated to one other server in the cluster and kept as a "passive Replica". Each server contains one partition and one replica of another server's partition. The replica is not directly accessed unless its main partition becomes unavailable for any reason (perhaps because its server has gone down). Then, this replica is made available as a replacement to its partition.
Partitioned-Replica uses asynchronous replication from the main partition to its replica. This ensures that replication is done very fast and without asking the user to wait for it. This allows the client to see the same type of performance as a Partitioned Cache even though data is being replicated to another server. Either way, the client is totally unaware of what's going on. It is able to perform its regular read and write operations without any interruptions.
Figure 6: Partition Cache with Replicas
Client Cache
If you have a clustered cache (any topology) as a separate caching tier, you can use Client Cache to keep a Local Cache close to your client that is a subset of the clustered cache and is synchronized with it. If anybody changes the same data in the clustered cache that the Client Cache has, the cluster cache notifies the Client Cache so it can either remove that item (if the item was also removed from the clustered cache) or reload the latest copy from the clustered cache.
A Client Cache is not part of the cache cluster. Instead, it is a Local Cache on the client end (be it local or remote client). It does however have a connection open with the clustered cache and the clustered cache notifies the Client Cache through NCache event notifications but only after the clustered cache has been updated successfully. So, there is a very small period after the clustered cache has been updated during which the Client Cache still has to be updated. The reason for keeping this separation is to allow a large number of clients to keep their own Client Caches but not degrade the clustered cache update performance.
Figure 7: Client Cache synchronized with clustered cache
Bridge Topology
If you want to provide disaster recovery (DR) in your infrastructure and have a different location data center designated as a DR site, your cache needs to replicate to the DR site as well. However, since the DR site is across the WAN, there is usually a great deal of latency between the primary and the DR sites and any replication ends up being extremely slow. So, if the DR site cache servers were made part of the primary site cache cluster, the entire cache cluster would grind to a halt due to this latency.
To address this problem, NCache provides a Bridge Topology. The Bridge consists of one or two servers that reside in the primary data center and the primary cache cluster is connected to it. All updates to the primary cluster are queued up on the Bridge which then updates the DR cache cluster asynchronously. The Bridge also has the intelligence to handle connection breakup which is common in WAN environment.
Figure 8: Bridge Topology replicating across WAN