Implement MapReduce Interface [Deprecated]

Users can implement the following MapReduce interfaces such as the IMapper, ICombinerFactory, ICombiner, IReducerFactory, IReducer, IKeyFilter, and more as detailed below.

Note

This feature is only available in NCache Enterprise.

Implement MapReduce Interface Prerequisites

Mapper

Implement the MapReduce interface IMapper and provide the implementation for the Map method. This method will contain the logic to map the input from the cache into more meaningful and goal-specific key-value pairs which can be sent to the Reducer or the optional Combiner. Referring to the workflow diagram, the string input is transformed by emitting each word with a key-value pair like <word, count> in the Mapper.

[Serializable]
public class ProductCountMapper : IMapper
{

    public void Map(object key, object value, IOutputMap context)
    {
        if (value is IList<string>)
        {
            IList<string> order = value as IList<string>;

            foreach (var product in order)
            {
                context.Emit(product, 1);
            }
        }
    }

    public void Dispose()
    {
        // Dispose resources
    }
}

Combiner Factory

Implement the MapReduce interface ICombinerFactory and provide the implementation for its Create(key) method. Providing the Combiner Factory or a Combiner is optional. This method will provide the incoming element with a new instance of the Combiner so that it merges the intermediate key-value pairs from the Mapper.

[Serializable]
public class ProductCountCombinerFactory : ICombinerFactory
{
    public IDictionary<string, ICombiner> combiners = new Dictionary<string, ICombiner>(StringComparer.CurrentCultureIgnoreCase);

    public ICombiner Create(object key)
    {
        if (!combiners.ContainsKey(key as string))
        {
            combiners.Add(key as string, new ProductCountCombiner());
        }

        return combiners[key as string];
    }
}
Note

To ensure the operation is fail-safe, it is recommended to handle any potential exceptions within your application, as explained in Handling Failures.

Combiner

Implement the MapReduce interface ICombiner. This method reduces the task result locally in chunks so that the Reducer is not burdened with excessive processing. If the workflow diagram is referred to, it means that the elements with the same keys are grouped before being sent to the Reducer for counting the word occurrence.

[Serializable]
public class ProductCountCombiner : ICombiner
{
    int count;

    public void BeginCombine()
    {
        //Any initialization related tasks

        count = 0;
    }

    public void Combine(object value)
    {
        if (value is int)
        {
            count += (int)value;
        }
    }

    public object FinishChunk()
    {
        // Finalize current chunk and
        // re -initialize resources here

        int countToBeReturned = count;
        count = 0;
        return countToBeReturned;
    }

    public void Dispose()
    {
        // Dispose resources
    }
}

Reducer Factory

Implement the MaoReduce interface IReducerFactory and provide the implementation for its Create(key) method. Providing the Reducer Factory or a Reducer is optional. This method will provide the incoming element with a new instance of the Reducer so that it merges the intermediate key-value pairs from the Combiner.

[Serializable]
public class ProductCountReducerFactory : IReducerFactory
{
    IDictionary<string, IReducer> reducers = new Dictionary<string, IReducer>(StringComparer.CurrentCultureIgnoreCase);

    public IReducer Create(object key)
    {
        if (!reducers.ContainsKey(key as string))
        {
            // If you want to map output against a different key,
            // specify a different key in constructor

            reducers.Add(key as string, new ProductCountReducer(key as string));
        }

        return reducers[key as string];
    }
}

Reducer

Implement the MapReduce interface IReducer. This method will reduce (process) the intermediate key-value pairs into further meaningful pairs. If the workflow diagram is referred to, it means that the values of the grouped elements from the Combiner are summed up to find the actual word count.

[Serializable]
public class ProductCountReducer : IReducer
{
    int count;
    string outputKey;
    KeyValuePair outputMapEntry;

    public ProductCountReducer(string key)
    {
        outputKey = key;
    }

    public void BeginReduce()
    {
        //All initialization related tasks

        count = 0;

        outputMapEntry = new KeyValuePair();

        // If you want to change keys in output map,
        // Change outputKey to your desired key

        outputMapEntry.Key = outputKey;
    }

    public void Reduce(object value)
    {
        if (value is int)
        {
            count += (int)value;
        }
    }

    public KeyValuePair FinishReduce()
    {
        outputMapEntry.Value = count;

        return outputMapEntry;
    }

    public void Dispose()
    {
        // Dispose resources
    }
}

Key Filter

Implement the MapReduce interface IKeyFilter to provide implementation for the FilterKey method. Providing this interface is optional. This method will allow the user to filter the input for Mapper by specifying the keys to be processed. If this option is not utilized, the whole cache data will be taken as input for the MapReduce tasks.

[Serializable]
public class ProductCountKeyFilter : IKeyFilter
{
    public bool FilterKey(object key)
    {
        // Execute map for keys containing word "Order"

        return (key as string).Contains("Order");
    }
}
Important

Make sure to deploy the MapReduce task libraries after implementing the interfaces using the NCache Management Center as guided in Configuring MapReduce in the Administrators' Guide.

Warning

This feature is only supported till 5.3 SP4.

See Also

Using MapReduce in Cache
Aggregator
Entry Processor
Configure MapReduce