MapReduce [Deprecated]

You can configure MapReduce for processing and generating large data sets with a parallel, distributed algorithm on a cluster.

Using the NCache Management Center

  • Launch the NCache Management Center by browsing to http://localhost:8251 or <server-ip>:8251 on Windows and Linux.

  • In the left navigation bar, click on Clustered Caches or Local Caches, based on the cache to configure.

  • Against the cache name, click on View Details.

Important

Ensure the cache stops before making any configuration changes.

View Details Mapreduce NCache Management Center

  • This opens the detailed configuration page for the cache. Go to the Advanced Settings tab and click on MapReduce in the left bar.

  • Configure the Max Tasks that can be executed simultaneously.

  • In case you expect exceptions to be thrown during task execution, you can specify Max avoidable exceptions, after which the task is failed and logged in the cache error log.

  • Specify Chunk Size - the number of emitted elements a chunk must have before being transmitted to the Combiner or Reducer.

  • Specify the Queue Size which is the maximum number of tasks that can wait in queue before they are processed.

  • Click Deploy Task Libraries.

  • A dialog box will open. Browse for the libraries that have the MapReduce interfaces implemented and click Open.

Mapreduce Web

  • Click on Save Changes to apply this configuration to the cache.

Using Command Line Tools

The Add MapReduce tool configures MapReduce tasks for processing and generating large data sets with a parallel, distributed algorithm on a clustered cache. The following command configures MapReduce execution on demoCache with its default options. The following command configures MapReduce on demoCache with 10 tasks to be executed in parallel with chunks of 100 elements each, 30 tasks to be enqueued and maximum 10 exceptions to be avoided.

Add-MapReduce -CacheName demoCache -MaxTasks 10 -ChunkSize 100 -QueueSize 30 -MaxExceptions 10

After configuring your MapReduce using the commands above, run the Install NCache Module command to deploy your provider assemblies.

See Also

Client Activity Events
Eviction Policy
Compression
Heartbeat