How do I read large data from NCache using streams?

You can retrieve large cache items by opening a CacheStream in Read or ReadWithoutLock mode using the GetCacheStream API, which provides a handle for sequential chunk-based reading.

What is the difference between Read and ReadWithoutLock modes?

Read mode allows multiple simultaneous readers but prevents any write operations, while ReadWithoutLock allows both read and write operations to occur in parallel on the same stream.

Retrieve Data from Streams [Deprecated]

In Streams Processing, the GetCacheStream API with overload is used to open a stream. It returns the handle of the opened stream as an instance of CacheStream class. To read data with streams, the streaming mode should be set to read mode.

In Streams Processing, the read mode can be acquired either with a lock or without a lock. In read with lock mode, multiple read operations can be performed simultaneously on a stream, but no write operation is allowed in this mode. While in read without lock, write operations can be done in parallel to read operations.

Prerequisites

Before using the NCache client-side APIs, ensure that the following prerequisites are fulfilled:

Install the following NuGet packages in your .NET client application:
- Enterprise: Alachisoft.NCache.SDK
- OpenSource: Alachisoft.NCache.Opensource.SDK
Include the following namespaces in your application:
- Alachisoft.NCache.Client
- Alachisoft.NCache.Runtime.Exceptions
The cache must be running.
Make sure that the data being added is serializable.
For API details, refer to: ICache, CacheStream, StreamMode, CacheStreamAttributes, GetCacheStream, Read.

Add the following Maven dependencies for your Java client application in pom.xml file:

<dependency>
    <groupId>com.alachisoft.ncache</groupId>
    <!--for NCache Enterprise-->
    <artifactId>ncache-client</artifactId>
    <version>x.x.x</version>
</dependency>

Import the following packages in your Java client application:
- import com.alachisoft.ncache.client.*;
- import com.alachisoft.ncache.runtime.exceptions.*;
The cache must be running.
Make sure that the data being added is serializable.
For API details, refer to: Cache, CacheStream, StreamMode, CacheStreamAttributes, getCacheStream, canRead, read, closed.

Install either of the following NuGet packages in your .NET client application:
- Enterprise: Install-Package Alachisoft.NCache.SDK -Version 4.9.1.0
Create a new Console Application.
Make sure that the data being added is serializable.
Add NCache References by locating %NCHOME%\NCache\bin\assembly\4.0 and adding Alachisoft.NCache.Web and Alachisoft.NCache.Runtime as appropriate.
Include the Alachisoft.NCache.Web.Caching namespace in your application.
To learn more about the NCache Legacy API, please download the NCache 4.9 documents available as a .zip file on the Alachisoft Website.

Read from Stream

This mode is used for acquiring read-only access to the stream. With the help of the example given below, a stream can be opened in read-only mode for the provided key.

Warning

The GetCacheStream with Read mode throws StreamNotFoundException if the specified key does not exist in the cache.

try
{
  // Precondition: Cache is already connected
  // Generate a unique cache key
  string key = "ncache-video";

  // Set StreamMode object to Read mode to read and write data with lock
  // StreamMode.Read allows only simultaneous reads
  // StreamMode.ReadWithoutLock allows both read and write
  StreamMode streamMode = StreamMode.ReadWithoutLock;

  // Provide the streamMode object to CacheStreamAttributes object
  var streamAttributes = new CacheStreamAttributes(streamMode);

  // Use GetCacheStream to initialize CacheStream object
  using (var cacheStream = cache.GetCacheStream(key, streamAttributes))
  {
      // Specify buffer to store data read from stream
      byte[] buffer = new byte[4096];

      while (true)
      {
          // Read data from stream
          int bytesRead = cacheStream.Read(buffer, 0, buffer.Length);

          // You can also write the incoming video data from the ongoing live event
          // Use the buffer data to stream the video live

          // Stop if all data is read from stream
          if (bytesRead == 0)
              break;
      }
  }
}
catch (OperationFailedException ex)
{
    // Exception can occur due to:
    // Connection Failures
    // Operation Timeout
    // Operation performed during state transfer
}
catch (StreamException ex)
{
    // Handle exception
}

try
{
  // Precondition: Cache is already connected
  // Generate a unique cache key
  int readByteCount = 0;
  byte[] readBuffer = new byte[5000];

  // Initialize CacheStream object
  CacheStream cacheStream = null;

  // StreamMode.Read allows only simultaneous reads but no writes!
  // StreamMode.ReadWithoutLock allows simultaneous reads and also lets the stream be writeable!
  CacheStreamAttributes cacheStreamAttributes = new CacheStreamAttributes(StreamMode.Read);

  using (CacheStream cacheStream = cache.getCacheStream(key, cacheStreamAttributes))
  {
      //int bytesRead = -1;
      while (cacheStream.CanRead)
      {
          var bytesRead = cacheStream.read(readBuffer, 0, readBuffer.length);
          readByteCount += bytesRead;

          // ReadBuffer is available for further processing/manipulation
          // Valid readBuffer contents (readBuffer[0] to readBuffer[bytesRead])
      }
  }

  Console.Writeline(key + " file content read as CacheStream. Total bytes: " + readByteCount);

  // Close stream
  cacheStream.Close();
}
catch (OperationFailedException exception) 
{
    // Exception can occur due to:
    // Connection Failures
    // Operation Timeout
    // Operation performed during state transfer
}
catch (Exception exception) 
{
    // Generic exception like IllegalArgumentException or NullPointerException
}

try
{
  // Using NCache Enterprise 4.9.1
  // Precondition: Cache is already connected
  string key = "ncache-video";
  // Open the cache stream in Read Mode
  using (CacheStream cacheStream = cache.GetCacheStream(key, StreamMode.Read))
  {
      byte[] buffer = new byte[4096];
      int bytesRead;
      long totalBytesRead = 0;
      while ((bytesRead = cacheStream.Read(buffer, 0, buffer.Length)) > 0)
      {
          // Here you can process the buffer (e.g., write to file or play video)
          totalBytesRead += bytesRead;
      }
  }
}
catch (OperationFailedException ex)
{
    // Exception can occur due to:
    // Connection Failures
    // Operation Timeout
    // Operation performed during state transfer
}
catch (StreamException ex)
{
    // Handle exception
}

Note

To ensure the operation is fail-safe, it is recommended to handle any potential exceptions within your application, as explained in Handling Failures.

Additional Resources

NCache provides sample application for Stream Processing on GitHub.