By leveraging MongoDB Change Streams, NCache provides real-time data consistency, which eliminates the risk of stale data in high-traffic distributed environments. Mongo DB is a NoSQL, scalable, open-source, high-performance, and document-oriented database. It is one of the best NoSQL databases in the market. Using caching with Mongo DB gives applications a significant performance boost by reducing trips to the database. However, caching Mongo DB data inside a distributed cache creates two copies of this data. One copy resides within Mongo DB, and one copy resides within the distributed cache. If an application changes the data inside Mongo DB directly, the data inside the cache becomes stale.
For high-transaction, multi-environment, and enterprise-level applications, this stale data is a big problem. In this blog, I’ll discuss how you can continue to use a distributed cache alongside Mongo DB and avoid the stale data problem at the same time.
Key Takeaways
Real-Time Data Integrity: Leverages MongoDB Change Streams (v3.6+) to provide a push-based notification system that eliminates data staleness in distributed caches.
Event-Driven Invalidation: Utilizes the NCache NotifyExtensibleDependency class to automatically remove or update cache items the moment a database change occurs.
Efficient Developer Experience: Implements the ICustomDependencyProvider interface, allowing for flexible, custom synchronization logic without heavy architectural overhead.
High-Availability Performance: Optimized for high-traffic environments like e-commerce and financial systems, where maintaining low-latency access to consistent data is mission-critical.
Synchronize Cache with Mongo DB Using NCache
NCache is a well-known distributed cache for .NET Framework/ .NET Core, Java, and Node.js. NCache provides a vast set of features that can improve the performance of your application by leaps and bounds. Among these vast sets of features is a feature known as Notification Based Extensible Dependency. This feature addresses the stale data problem with MongoDB (which we discussed earlier).
Figure 1: NCache and MongoDB Synchronization Workflow.
The NotifyExtensibleDependency class allows you to write your custom dependency logic in which you are notified of the changes taking place within the database through notifications. NCache processes these notifications by making use of a feature of Mongo DB (introduced in version 3.6) known as Change Stream. Once these notifications are processed, stale data within the cache is invalidated based on the code you have written.
The Change Stream logic is integrated inside the NotifyExtensibleDependency. You can modify this Change Stream to receive notifications on add, update, or delete operations that take place inside the database. But, the delete operations can’t be tracked. So, to track delete operations, an update operation with expiry is used.
It should also be kept in mind that Mongo DB doesn’t support Change Stream for Stand-Alone databases. You need to ensure that your Mongo DB database has a replica set so that the Change Stream feature works.
To ensure high performance, it is vital to compare event-driven synchronization with traditional methods. The table below highlights why MongoDB Change Streams is the superior choice for maintaining real-time cache consistency:
| Feature | MongoDB Change Streams (Recommended) | Traditional Database Polling |
|---|---|---|
| Communication | Push-based: MongoDB alerts NCache immediately. | Pull-based: NCache must “ask” the DB for updates. |
| Latency | Real-time: Near-instant cache invalidation. | Delayed: Dependent on the polling interval. |
| Performance | High Efficiency: Minimal overhead on the database. | Lower Efficiency: Constant queries can strain DB resources. |
| Scalability | Excellent: Handles high-traffic environments easily. | Limited: Polling frequency becomes a bottleneck. |
| Implementation | Uses NotifyExtensibleDependency. |
Uses custom timer-based logic. |
NCache Details NCache NotifyExtensibleDependency NCache Docs
Sync Cache with Mongo DB: A Quick Example
To use Mongo DB with NCache you need to configure Custom Dependency in NCache first and then deploy it in your client application afterward. This is explained below:
NCache Custom Dependency Configuration
To use the Notification Based Extensible Dependency feature of NCache with Mongo DB, you need to configure Custom Dependency on NCache first and deploy your respective DLL file(s) on the NCache Server.
In the GIF below, I have deployed a provider named MongoDbNotifyExtensibleDependencyProvider along with all of its dependencies on the cache. It implements the ICustomDependencyProvider interface. It is responsible for creating a dependency object for the CacheItem. For the dependency to be triggered properly, the NotifyExtensibleDependency class must be implemented. This class will listen to the MongoDB Change Stream and trigger dependencies upon data change in Mongo DB.
Also, in this example, I am using an open-source implementation of ICustomDependencyProvider and NotifyExtensibleDependency available on GitHub.
Figure 2: Notify Extensible Dependency Provider Deployment on NCache
NCache Details Custom Dependency Configuration GitHub Repository
Deployment on Client Application
For example, you wish to use Mongo DB as a database for your E-Commerce store and want to cache frequently used data inside NCache. Your store will contain data of over a million products, and millions of transactions will take place daily. NCache will help with reducing the load on your Mongo DB database, but the data inside the cache is prone to going stale.
Having stale data inside the cache can cause problems for you and your customers. Fortunately, the Notification Based Extensible Dependency feature of NCache mitigates the problem of stale data ensuring the optimal functionality of your E-Commerce store.
In the following example, a product is loaded from the database based on its ID. Then NotifyExtensibleDependency is created and registered against the fetched product. Finally, the product, in the form of CacheItem is inserted into the cache with the key specified.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
Product product = LoadProductFromDatabase(productId); //This is the name of your provider deployed on a cache server string providerName = "MongoDbNotifyExtensibleDependencyProvider"; string key = "Product:" + product.Id ; IDictionary<string, string> parameters = new Dictionary<string, string>(); parameters.Add("ConString", "mongodb://**.**.**.**:27017"); parameters.Add("DatabaseName", "productscollection"); parameters.Add("CollectionName", "products"); CacheItem item = new CacheItem(product); item.Dependency = new CustomDependency(providerName, parameters); _cache.Insert(key, item); |
NCache Details NotifiyExtensibleDependency Cache Operations in NCache
Using Read-Through Provider
Another prominent feature of NCache is the Read-Through Provider. This feature allows you to look for data inside Mongo DB directly when it isn’t found in the cache. When the item is found inside the database, it is also stored inside the cache automatically; thus, saving you precious time. You can use this feature in your Mongo DB applications to further enhance your performance.
The following code example retrieves an item with read-through enabled, corresponding to the specified key “Product:1001” using the Get<> method.
|
1 2 3 4 5 6 7 8 9 |
// Specify the key of the item string key = "Product:1001"; // Specify the readThruOptions for read through operations var readThruOptions = new ReadThruOptions(); readThruOptions.Mode = ReadMode.ReadThru; // Retrieve the data of the corresponding item with reads thru enabled Product product = cache.Get(key, readThruOptions); |
NCache Details Read Through Caching Cache Operations in NCache
Conclusion
To sum up, NCache offers a flexible solution for synchronizing data between Mongo DB and the cache, thereby avoiding data inconsistency. NCache, an in-memory distributed caching solution, can be an ideal medium for Mongo DB to maintain up-to-date data for fast access and processing.
NCache Details Download NCache Edition Comparison
Frequently Asked Questions (FAQ)
How does NCache stay synchronized with MongoDB?
NCache maintains synchronization by utilizing a custom extensible dependency that monitors MongoDB Change Streams. When a change occurs in the database (Insert, Update, or Replace), MongoDB pushes a notification to NCache. The cache then uses the –class to immediately invalidate or update the corresponding entry, ensuring the distributed cache remains consistent with the primary database.
What is the role of NotifyExtensibleDependency in MongoDB synchronization?
The NotifyExtensibleDependency class is the core component that allows NCache to react to external database events. It acts as a listener that stays active within the cache cluster; once it receives a trigger from the MongoDB Change Stream, it executes the invalidation logic. This replaces the need for resource-intensive polling and provides a more reactive, event-driven architecture.
Which MongoDB versions support NCache real-time synchronization?
Real-time synchronization with NCache requires MongoDB version 3.6 or higher. This is because the solution relies specifically on the Change Streams feature introduced in that version. Earlier versions of MongoDB do not support the push-based notification framework necessary for this event-driven invalidation model.
How is the synchronization provider deployed in NCache?
The synchronization logic is implemented through the ICustomDependencyProvider interface. To deploy it, you must compile your logic into a DLL file and register it as a Custom Dependency in the NCache Management Center. This deployment allows the NCache server to natively handle the connection to MongoDB and manage the lifecycle of the change stream notifications.
