• Webinars
  • Docs
  • Download
  • Blogs
  • Contact Us
Try Free
Show / Hide Table of Contents

Cache Data Dependency on File

Note

This feature is only available in NCache Enterprise Edition.

File dependency provides a mechanism to invalidate the cache data in case of non-relational data sources. Here you can associate a file/folder dependency with a cache item. Whenever that file/folder is removed or modified, NCache will remove its dependent item from cache. Cache clean up thread monitors the dependent file/folder for any change at every CleanInterval.

On cache clean up thread, NCache triggers dependency in the following scenarios:

  • Dependency file is removed/modified.
  • Dependency folder is removed/modified.
  • Dependency is created in the non-existed file/folder, but on the cleanup interval it is created.

Similarly, multiple items can depend on a single file. Any change in file, either by updating the file or deletion, causes the cache to remove the dependent items from the cache. Likewise, an item can create dependency on multiple files in file dependency.

You can also provide a delay called startAfter in file dependency which indicates when to start monitoring the dependent file for any change. In this way NCache will start checking the dependent file after the startAfter time has elapsed.

Prerequisites

  • .NET/.NET Core
  • Java
  • Scala
  • Node.js
  • Python
  • To learn about the standard prerequisites required to work with all NCache client side features please refer to the given page on Client Side API Prerequisites.
  • For API details, refer to: ICache, CacheItem, CacheItemAttributes, Dependency, UpdateAttributes, FileDependency.
  • To learn about the standard prerequisites required to work with all NCache client side features please refer to the given page on Client Side API Prerequisites.
  • For API details, refer to: Cache, CacheItem, CacheItemAttributes, setDependency, updateAttributes, FileDependency.
  • To learn about the standard prerequisites required to work with all NCache client side features please refer to the given page on Client Side API Prerequisites.
  • For API details refer here: Cache, CacheItem, CacheItemAttributes, setDependency, FileDependency.
  • To learn about the standard prerequisites required to work with all NCache client side features please refer to the given page on Client Side API Prerequisites.
  • For API details, refer to : Cache, CacheItem, CacheItemAttributes, setDependency, updateAttributes, FileDependency.
  • To learn about the standard prerequisites required to work with all NCache client side features please refer to the given page on Client Side API Prerequisites.
  • For API details refer to: Cache, CacheItem, CacheItemAttributes, set_dependency, update_attributes, FileDependency.
Important

Make sure that the NCache service has access to the file path where the file to be dependent on is placed.

Add Data to Cache with File Dependency

CacheItem is a custom class provided by NCache which can be used to add data to the cache and also lets you set additional metadata associated with an object of this class. This metadata defines the properties of the item like dependencies, expirations and more.

The Add method adds a new item in the cache whereas the Insert method adds a new item with dependency and if the item already exists in the cache it overwrites its properties.

Important

Note that this API also specifies cache item priority for eviction as well as expiration, so the value for that parameter has been passed as Default, as it is not discussed here.

In the following example, the data added in the cache is the content of the file placed at the specified path as a CacheItem with dependency on the file. Any change in the file will result in the expiration of the dependent data present in the cache. The expired items will then be removed at the cache CleanInterval.

  • .NET/.NET Core
  • Java
  • Scala
  • Node.js
  • Python
try
{
    // Pre-Condition: Cache is already connected

    // Generate a unique key for the fileData
    string key = "FileData";

    // Get the content of the file as string
    string FileData = cache.Get<string>(key);

    // Check if the fileData is already in the cache
    if (string.IsNullOrEmpty(FileData))
    {
        // Read the contents of the file placed at the path
        using (var streamReader = new StreamReader(@"D:\\Dependency.txt"))
        {
            FileData = streamReader.ReadToEnd();
        }
    }
    // Specify the file path to add dependency on the file
    string filepath = "D:\\Dependency.txt";

    // Create a new cacheItem with filedata
    var cacheItem = new CacheItem(FileData);

    // Create the dependency on the file on the specified path
    cacheItem.Dependency = new FileDependency(filepath);

    // Add the file data in the cache with dependency on the file
    cache.Insert(key, cacheItem);
}
catch (OperationFailedException ex)
{
    // NCache specific exception
    // Exception can occur due to:
    // Connection Failures
    // Operation Timeout
    // Operation performed during state transfer
}
catch (Exception ex)
{
    // Any generic exception like ArgumentNullException or ArgumentException
}
try{
    // Precondition: Cache is already connected

    // Generate a unique key for the fileData
    String key = "FileData";

    // Get the content of the file as string
    String fileData = cache.get(key, String.class);

    // Check if the fileData is already in the cache
    if (fileData ! = null){
        // Read the contents of the file placed at the path
        File file = new File("test.txt");
        try (BufferedReader buffer = new BufferedReader(new FileReader(file))){
            buffer.readLine();
        }
    }

    // Specify the file path to add dependency on the file
    String filePath = "test.txt";

    // Create a new cache item with the file data
    CacheItem cacheItem = new CacheItem(fileData);

    // Create file dependency on the file placed at filePath
    cacheItem.setDependency(FileDependency(filePath));

    // Add the file data in the cache with dependency on the file
    cache.insert(key, cacheItem);
}catch (OperationFailedException ex){
    // NCache specific exception
    // Exception can occur due to:
    // Connection Failures
    // Operation Timeout
    // Operation performed during state transfer
}catch (Exception ex){
    // Any generic exception like IllegalArgumentException or NullPointerException
}
try {
    // Precondition: Cache is already connected

    // Generate a unique key for the fileData
    val key = "FileData"

    // Get the content of the file as string
    var fileData = cache.get(key, classOf[String])

    // Check if the fileData is already in the cache
    if (fileData != null) {
      // Read the contents of the file placed at the path
      var file = new File("test.txt")
      try {
        var buffer = BufferedReader(FileReader(file))
        buffer.readLine()
      } catch {
        case exception: Exception => // Handle exceptions
      }
    }

    // Specify the file path to add dependency on the file
    val filePath = "test.txt"

    // Create a new cache item with the file data
    var cacheItem = CacheItem(fileData)

    // Create file dependency on the file placed at filePath
    cacheItem.setDependency(FileDependency(filePath))

    // Add the file data in the cache with dependency on the file
    cache.insert(key, cacheItem)
}
catch {
    case exception: Exception => {
      // Handle any errors
    }
}
try {
    // Precondition: Cache is already connected

    // Generate a unique key for the fileData
    let key = "FileData";

    let filePath = "test.txt";

    // Get the content of the file as string
    let fileData = this.cache.get(key, String.class);

    let blob = new Blob;
    // Read the contents of the file placed at the path

    if (!(fileData == null)) {
        var file = new FileReader();
        file.readAsText(blob, filePath);
    }

    // Create a new cache item with the file data
    let cacheItem = new ncache.CacheItem(fileData);

    // Create file dependency on the file placed at filePath
    cacheItem.setDependency(new ncache.FileDependency(filePath));

    // Add the file data in the cache with dependency on the file
    this.cache.insert(key, cacheItem);

} catch (error) {
    // Handle any errors
}
try:
    # Precondition: Cache is already connected

    # Generate a unique key for the fileData
    key = "FileData"

    file_path = "Path/To/File"

    # Get the content of the file as string
    file_data = cache.get(key, str)

    # Check if the file_data is already in the cache
    if file_data is None or not file_data:
        # Read the contents of the file placed at the path
        file = open(file_path, mode='r')
        file_data = file.read()
        file.close()

    # Create a new cache item with the file data
    cache_item = ncache.CacheItem(file_data)

    # Create file dependency on the file placed at filePath
    cache_item.set_dependency(ncache.FileDependency(file_path))

    # Add the file data in the cache with dependency on the file
    cache.insert(key, cache_item)
except Exception as exp:
    # Handle errors
Note

To ensure the operation is fail safe, it is recommended to handle any potential exceptions within your application, as explained in Handling Failures.

Trigger File Dependency with Delay

If you want to add dependency to an item which is triggered after a particular time, NCache allows you to specify the time interval for that.

The Add method adds a new item in the cache whereas the Insert method adds a new item with dependency and if the item already exists in the cache it overwrites its properties.

Important

Note that this API also specifies cache item priority for eviction as well as expiration, so the value for that parameter has been passed as Default, as it is not discussed here.

In the following example, the data added in the cache is the content of the file placed at the specified path with dependency on the file. The dependency will be triggered after a span of 20 minutes. Any change in the file will result in the expiration of the dependent data present in the cache. The expired items will then be removed at the cache CleanInterval.

  • .NET/.NET Core
  • Java
  • Scala
  • Node.js
  • Python
// Generate a unique key for the fileData
string key = "FileData";

// Get the content of the file as string
string FileData = cache.Get<string>(key);

// Check if the fileData is already in the cache
if (string.IsNullOrEmpty(FileData))
{
    // Read the contents of the file placed at the path
    using (var streamReader = new StreamReader(@"D:\\Dependency.txt"))
    {
        FileData = streamReader.ReadToEnd();
    }
}
// Specify the file path to add dependency on the file
string filepath = "D:\\Dependency.txt";

// Create a new CacheItem with the key
var cacheItem = new CacheItem(FileData);

// Create the dependency on the file on the specified path
// The dependency starts after 20 minutes
cacheItem.Dependency = new FileDependency(filepath, DateTime.Now.AddMinutes(20));
cache.Insert(key, cacheItem);
// Generate a unique key for fileData
String key = "FileData";

// Get the content of the file as string
String fileData = cache.get(key, String.class);

// Check if the file is already in the cache
if (fileData ! = null){
    // Read the contents of the file placed at the path
    File file = new File("test.txt");
    try (BufferedReader buffer = new BufferedReader(new FileReader(file))){
        buffer.readLine();
    }
}

// SPecify the file path to add dependency to the file
String filePath = "test.txt";

// Create a new cache item with the key
CacheItem cacheItem = new CacheItem(fileData);

// Create delay for dependency
Calendar now = Calendar.getInstance();
now.add(Calendar.MINUTE, 20);
Date delayTime = now.getTime();

// Create file dependency on the file placed at filePath
// The dependency starts after 20 minutes
cacheItem.setDependency(FileDependency(filePath, delayTime));

// Add the file data in the cache with dependency on the file
cache.insert(key, cacheItem);
// Generate a unique key for the fileData
val key = "FileData"

// Get the content of the file as string
var fileData = cache.get(key, classOf[String])

// Check if the fileData is already in the cache
if (fileData != null) {
    // Read the contents of the file placed at the path
    var file = File("test.txt")
    try {
    var buffer = BufferedReader(FileReader(file))
    buffer.readLine()
    } catch {
    case exception: Exception => // Handle exceptions
    }
}

// Specify the file path to add dependency on the file
val filePath = "test.txt"

// Create a new cache item with the file data
var cacheItem = CacheItem(fileData)

// Create delay for dependency
val now = Calendar.getInstance
now.add(Calendar.MINUTE, 20)
val delayTime = now.getTime

// Create file dependency on the file placed at filePath
// The dependency starts after 20 minutes
cacheItem.setDependency(FileDependency(filePath, delayTime));

// Add the file data in the cache with dependency on the file
cache.insert(key, cacheItem)
// Generate a unique key for the fileData
let key = "FileData";

let filePath = "test.txt";

// Get the content of the file as string
let fileData = this.cache.get(key, String.class);

let blob = new Blob;
// Read the contents of the file placed at the path

if (!(fileData == null)) {
    var file = new FileReader();
    file.readAsText(blob, filePath);
}

// Create a new cache item with the key
let cacheItem = new ncache.CacheItem(fileData);

// Create delay for dependency
let now = ncache.Calender.getInstance();
now.add(ncache.Calendar.MINUTE, 20);
let delayTime = now.getTime();

// Create file dependency on the file placed at filePath
// The dependency starts after 20 minutes
cacheItem.setDependency(new ncache.FileDependency(filePath, delayTime));



// Add the file data in the cache with dependency on the file
this.cache.insert(key, cacheItem);
# Generate a unique key for the fileData
key = "FileData"

# Specify the file path to add dependency on the file
file_path = "Path/To/File"

# Get the content of the file as string
file_data = cache.get(key, str)

# Check if the file_data is already in the cache
if file_data is None or not file_data:
    # Read the contents of the file placed at the path
    file = open(file_path, mode='r')
    file_data = file.read()
    file.close()

# Create a new CacheItem with the file_data
cache_item = ncache.CacheItem(file_data)

# Create the dependency on the file on the specified path
# The dependency starts after 20 minutes
cache_item.set_dependency(ncache.FileDependency(file_path, datetime.now() + timedelta(minutes=20)))
cache.insert(key, cache_item)

Add File Dependency on Multiple Files

You can also add file dependency to an item which is dependent on multiple files or folders. This way a single item can be dependent on multiple items using the Add or Insert method. The Add method adds a new item in the cache whereas the Insert method adds a new item with dependency and if the item already exists in the cache it overwrites its properties.

Important

Note that this API also specifies cache item priority for eviction as well as expiration, so the value for that parameter has been passed as Default, as it is not discussed here.

Following example adds an item with key "1001" which is dependent on two files placed on the specified paths.

  • .NET/.NET Core
  • Java
  • Scala
  • Node.js
  • Python
// Get product from database against given ProductID
Product product = FetchProductByProductID(1001);

    // Create a unique cache key for this product
String key = "Product:" + product.ProductID;

// Create a new CacheItem product
CacheItem cacheItem = new CacheItem(product);

// Specify the filepaths of the master files
// Make sure that the network file paths are accessible by NCache Service
string[] fileNames = new string[1];
fileNames[0] = "\\fileserver1\\ProductList.csv";
fileNames[1] = "\\fileserver1\\OrderList.csv";

//Adding cache item "Product:1001" with FileDependency on a list of files
cacheItem.Dependency = new FileDependency(fileNames);

cache.Insert(key, cacheItem);

// For successful addition of item with dependency
// Update or remove the file
// Verify if key is present
// Get product from database against productId
Product product = fetchProductFromDb(productId);

// Create a unique cache key for this product
String key = "Product:" + product.productId;

// Create a new cache item
CacheItem cacheItem = new CacheItem(product);

// SPecify paths of the master files
// Make sure that the network file paths are accessible by NCache Service
fileNames[0] = "\\fileserver1\\ProductList.csv";
fileNames[1] = "\\fileserver1\\OrderList.csv";

// Adding file dependency on the specified cache item
cacheItem.setDependency(FileDependency(Arrays.asList(fileNames)));

// Insert the cache item with file dependency
cache.insert(key, cacheItem);

// For successful addition of item with dependency
// Update or remove the file
// Verify if key is present
val productId = "1001"

// Get product from database against productId
val product = fetchProductFromDb(productId)

// Create a unique cache key for this product
val key = "Product:" + product.getProductId

// Create a new cache item
val cacheItem = CacheItem(product)

// Specify paths of the master files
// Make sure that the network file paths are accessible by NCache Service
val fileNames = List("\\fileserver1\\ProductList.csv", "\\fileserver1\\OrderList.csv")

// Adding file dependency on the specified cache item
cacheItem.setDependency(FileDependency(fileNames))

// Insert the cache item with file dependency
cache.insert(key, cacheItem)

// For successful addition of item with dependency
// Update or remove the file
// Verify if key is present
// Get product from database against productId
let product = this.fetchProductFromDb(this.productId);

// Create a unique cache key for this product
let key = "Product:" + product.ProductID;

// Create a new cache item
let cacheItem = new ncache.CacheItem(product);

// SPecify paths of the master files
// Make sure that the network file paths are accessible by NCache Service
let fileNames = ["\\fileserver1\\ProductList.csv", "\\fileserver1\\OrderList.csv"];

// Adding file dependency on the specified cache item
cacheItem.setDependency(new ncache.FileDependency(fileNames));

// Insert the cache item with file dependency
this.cache.insert(key, cacheItem);

// For successful addition of item with dependency
// Update or remove the file
# Get product from database against productId
product = fetch_product_from_db("1001")

# Create a unique cache key for this product
key = "Product:" + product.get_product_id()

# Create a new cache item
cache_item = ncache.CacheItem(product)

# Specify paths of the master files
# Make sure that the network file paths are accessible by NCache Service
file_names = ["\\fileserver1\\ProductList.csv", "\\fileserver1\\OrderList.csv"]

# Adding file dependency on the specified cache item
cache_item.set_dependency(ncache.FileDependency(file_names))

# Insert the cache item with file dependency
cache.insert(key, cache_item)

# For successful addition of item with dependency
# Update or remove the file

Add File Dependency to Existing Cache Items

NCache also provides you with the ease of adding key dependency to an item already present in cache, without re-inserting it into the cache.

Important

This approach is comparatively less lightweight and more cost efficient since items are already present in the cache.

Using UpdateAttributes() API

This is done through CacheItemAttribute class, which has the property of Dependency to be set against CacheItem. The attribute is then set against the existing key of the item, using the UpdateAttributes method of Cache class.

The following example adds an item to cache without dependency, and then sets the file dependency for the item using UpdateAttributes method. This requires no need to add the item again to the cache.

  • .NET/.NET Core
  • Java
  • Scala
  • Node.js
  • Python
// Generate a unique key for the fileData
string key = "FileData";

// Get the content of the file as string
// Make sure that the item already exists in the cache
string FileData = cache.Get<string>(key);

// Specify the filepath
string filepath = "\\FileHost\\ProductList.csp";

// Create a file dependency where key is dependent on the file on specified path
var dependency = new FileDependency(filepath);

// Create a CacheItemAttribute for dependency
var attr = new CacheItemAttributes();
attr.Dependency = dependency;

// Set the attribute of dependency against key
cache.UpdateAttributes(key, attr);

// Monitor/Verify dependency through either:
// PerfMon Counters or Cache API
// Generate a unique cache key for file data
String key = "FileData";

// Get the content of the file as string
// Make sure that the item already exists in the cache
String fileData = cache.get(key, String.class);

// Specify the filepath
String filePath = "\\FileHost\\ProductList.csp";

// Create a file dependency where key is dependent on the file placed at filePath
var dependency = new FileDependency(filePath);

// Create a cache item attribute for dependency
CacheItemAttributes attribute = new CacheItemAttributes();
attribute.setDependency(dependency);

// Set the attributes of dependency against the existing cache key
cache.updateAttributes(key, attribute);

// Monitor/Verify dependency through either:
// PerfMon Counters or Cache API
// Generate a unique cache key for file data
val key = "FileData"

// Get the content of the file as string
// Make sure that the item already exists in the cache
val fileData = cache.get(key, classOf[Nothing]).toString

// Specify the filepath
val filePath = "\\FileHost\\ProductList.csp"

// Create a file dependency where key is dependent on the file placed at filePath
val dependency = FileDependency(filePath)

// Create a cache item attribute for dependency
val attribute = CacheItemAttributes()
attribute.setDependency(dependency)

// Set the attributes of dependency against the existing cache key
cache.updateAttributes(key, attribute)

// Monitor/Verify dependency through either:
// PerfMon Counters or Cache API
// Generate a unique cache key for file data
let key = "FileData";

// Get the content of the file as string
// Make sure that the item already exists in the cache
let fileData = this.cache.get(key, String.class);

// Specify the filepath
let filePath = "\\FileHost\\ProductList.csp";

// Create a file dependency where key is dependent on the file placed at filePath
var dependency = new ncache.FileDependency(filePath);

// Create a cache item attribute for dependency
var attribute = new ncache.CacheItemAttributes();
attribute.setDependency(dependency);

// Set the attributes of dependency against the existing cache key
this.cache.updateAttributes(key, attribute);

// Monitor/Verify dependency through PerfMon Counters or Cache API
# Generate a unique cache key for file data
key = "FileData"

# Specify the filepath
file_path = "\\FileHost\\ProductList.csp"

# Create a file dependency where key is dependent on the file placed at file_path
dependency = ncache.FileDependency(file_path)

# Create a cache item attribute for dependency
attribute = ncache.CacheItemAttributes()
attribute.set_dependency(dependency)

# Set the attributes of dependency against the existing cache key
cache.update_attributes(key, attribute)

# Monitor / Verify dependency through PerfMon Counters or Cache API

Additional Resources

NCache provides sample application for File Dependency on GitHub.

See Also

Custom Dependency
Cache Data Dependency on Database
Data Dependency for Relationship in Cache

Back to top Copyright © 2017 Alachisoft