NCache 4.6 - Online Documentation

Sample Implementation of MapReduce Interfaces

 
      To utilize the following interfaces, include the following namespace in your application: Alachisoft.NCache.Runtime.MapReduce.
 
Mapper
Implement the interface IMapper and provide implementation for its Map() method.
 
Member
Description
Map()
This method will contain the logic to map the input from cache into more meaningful and goal specific key-value pairs which can be sent to the Reducer or optional Combiner. Referring to the workflow diagram, the string input is transformed by emitting each word with a key-value pair like <word, count> in the Mapper.
 
  public class WordCountMapper : IMapper
    {
        string[] parsedline;
        string line;
 
        public void Map(Object key, Object value, IOutputMap outputMap)
        {
            line = value.ToString();
 
            parsedline = line.Split(' ');
 
            for (int i = 0; i < parsedline.Length; i++)
            {
                outputMap.Emit(parsedline[i], 1);
            }
        }
 
        public void Dispose()
        {
          //dispose resources
        }
    }
 
 
Combiner Factory
Implement the interface ICombinerFactory and provide implementation for its Create(key) method. Providing Combiner Factory or Combiner is optional.
 
Member
Description
Create(key)
This method will provide the incoming element with a new instance of the Combiner so that it merges the intermediate key-value pairs from the Mapper.
 
  public class WordCountCombinerFactory : ICombinerFactory
    {
        public ICombiner Create(object key)
        {
            WordCountCombiner wcCombiner = new WordCountCombiner();
            return wcCombiner;
        }
    }
 
 
Combiner
Implement the interface ICombiner and provide implementation for the methods:
 
Member
Description
BeginCombine()
This method will provide the user with a starting point to initialize any parameters you want to be used before the actual combining of elements begins.
Combine()
This method reduces the task result locally in chunks so that the Reducer is not burdened with excessive processing. If the workflow diagram is referred, it means that the elements with the same keys are grouped together before being sent to the Reducer for counting the word occurrence.
FinishChunk()
This method marks the end of the Combiner functionality as when the number of combined results reaches the specified chunk size, it sends the tasks in the form of a chunk to the Reducer and resets its internal state for the next chunk.
 
  public class WordCountCombiner : ICombiner
    {
        int count = 0;
 
        public void BeginCombine()
        {
            //any initialization
        }
 
        public void Combine(object value)
        {
            count += int.Parse(value.ToString());
        }
 
        public object FinishChunk()
        {
            return count;
        }
 
        public void Dispose()
        {
            //dispose resources
        }
    }
 
 
Reducer Factory
Implement the interface IReducerFactory and provide implementation for its Create(key) method. Providing Reducer Factory or Reducer is optional.
 
Member
Description
Create(key)
This method will provide the incoming element with a new instance of the Reducer so that it merges the intermediate key-value pairs from the Combiner.
 
public class WordCountReducerFactory : IReducerFactory
    {
        public IReducer Create(object key)
        {
            WordCountReducer wcReducer = new WordCountReducer(key);
            return wcReducer;
        }
    }
 
 
Reducer
Implement the interface IReducer and provide implementation for the methods:
 
Member
Description
BeginReduce()
This method will provide the user with a starting point to initialize any parameters you want to be used before the actual reducing of elements begins.
Reduce()
This method will reduce (process) the intermediate key-value pairs into further meaningful pairs. If the workflow diagram is referred, it means that the values of the grouped elements from Combiner are summed up to find the actual word count.
FinishReduce()
This method will provide the task with the final result of the map reduce operation for specific intermediate key/value pair.
 
  public class WordCountReducer : IReducer
    {
        int count = 0;
        object reducerKey;
 
        public WordCountReducer(object key)
        {
            reducerKey = key;
        }
 
        public void BeginReduce()
        {
            //perform operations
        }
 
        public void Reduce(object value)
        {
            count += int.Parse(value.ToString());
        }
 
        public KeyValuePair FinishReduce()
        {
            KeyValuePair kvp = null;
            kvp.Key = reducerKey;
            kvp.Value = count;
 
            return kvp;
        }
 
        public void Dispose()
        {
            //dispose resources
        }
    }
 
 
Key Filter
Implement the interface IKeyFilter to provide implementation for the FilterKey() method. Providing this interface is optional.
 
Member
Description
FilterKey(key)
This method will allow user to filter the input for Mapper by specifying the keys to be processed. If this option is not utilized, the whole cache data will be taken as input for the MapReduce tasks.
 
  public class WordCountKeyFilter : IKeyFilter
    {
        public bool FilterKey(object key)
        {
            try
            {
                if (key.ToString().Contains("hungry"))
                {
                    return true;
                }
 
            }
            catch (Exception exp)
            {
                //handle exception
            }
            return false;
        }
 
 
Task Tracker
 
You do not need to implement this component.
 
The ITrackableTask interface implements the following members, which can be used if you want to track the execution of your task:
 
Member
Description
event MapReduceCallback OnMapReduceComplete
Users can register a callback on the MapReduce task that is called when task execution is completed, failed or cancelled with a parameter response that encapsulates status of the task and result (if completed).
GetResult()
This is a blocking call that waits for the callback from the server about the task’s completion, failure or cancellation in the form of an ITaskResult. The ITaskResult lets you get the following:
·     GetEnumerator: obtain the result in the form of a dictionary
·     Status: will return either failure, success, cancelled.
·     TaskFailureReason: the reason behind the failure of the task.
GetResult(timeout)
If no result is obtained within the specified timeout duration, OperationFailedException is thrown.
TaskId
This is a GUID identification ID of the task to mark it as unique.
TaskStatus()
Gets the task’s status and may contain one of the following values:
Waiting, InProgress, Completed, Cancelled, Failed
CancelTask()
 
To cancel the already running task.
 
MapReduceCallback and GetResult() cannot be executed together because getResult is a blocking call and this combination will throw an exception.
 
Make sure to deploy the task libraries after implementing the interfaces using NCache Manager.
 
 
See Also