Let’s say you pay at a mart using your card. Now you are at home, haven’t gone out in a while and you receive a message from your bank stating you have debited $20000. How? What you purchased cost you a mere $120! This will entail a long process of verifications with your bank, and the bank may take months in tracing and resolving the fraudulent transaction.
It would be much easier if a fraud detection mechanism is incorporated in the system so that it determines invalid transactions and fails them in real-time. With global payment fraud losses expected to exceed $40 billion annually, the cost of delay is no longer sustainable. However, this decision has to be instant and the system should train itself continually based on the user’s past behaviors. Hence, you can’t expect customers to wait at the counter while your fraud detection system takes time in detecting any anomaly. You need to have an efficient as well as faster system. This is possible if you store the data in-memory using a cache such as NCache and process it within milliseconds.
Key Takeaways:
Sub-Millisecond Processing: NCache eliminates database bottlenecks by storing data in-memory, allowing fraud detection to occur within milliseconds during a live transaction.
Decoupled Architecture: Using NCache as a Pub/Sub Messaging Bus separates transaction management from fraud logic, ensuring high throughput and system scalability.
Three-Tiered Logic: The system balances instant detection (NRT System) with long-term accuracy by feeding data into Offline-Processing for continuous ML model training.
Stateful Tracking: Distributed Data Structures (like Lists) maintain real-time customer behavior history, enabling instant pattern recognition and anomaly detection.
Non-Blocking Persistence: Write-Behind caching ensures the database is updated asynchronously, preventing disk I/O from slowing down the real-time user experience.
Figure 1: High-Performance Event-Driven Fraud Detection Architecture powered by NCache
Generally, a fraud detection system contains three subsystems, as illustrated in Figure 1:
- Near Real-Time (NRT) System: This system processes the input and the results are generated using pre-defined patterns and rules, instead of performing the actual calculation. Moreover, the results must be in milliseconds as it responds directly to the user. The main purpose of this system is high throughput with low latency which means this system should ideally be in local memory to achieve higher performance.
- Input-Processing System: This system can take up to minutes to hours for processing the input and adjusting the parameters of the NRT system. This updated information is fed to the offline processing system, that further processes machine learning algorithms on it.
- Offline-Processing System This system can take from hours to months, to respond. This system improves the model, trains the data, and feeds this upgraded data back to the NRT system for better results.
NCache as In-memory Cache and Pub/Sub Messaging Bus
NCache is an in-memory distributed cache that offers powerful Pub/Sub messaging and other features such as distributed data structures. Keeping in view the systems involved in a fraud detection system, NCache seamlessly fits in the Near-Real-Time System as it requires blazing fast results. This is possible as NCache can be used for communication between multiple elements as well as storing data nearest to your application, further reducing network calls to fetch data.
We have created a sample fraud detection system in .NET that includes NCache to improve performance and throughput. You can find the working sample here at GitHub.
At an upper level, the application works as such:
- The client application communicates with the Transaction Request Manager that manages the transaction of the given customer.
- The transaction information is published as a message to the Pub/Sub topic.
- The Fraud Detection Manager subscribes to this topic and receives the transaction message.
- The fraud detection logic is performed on the item and the result is sent to the transaction manager that is sent back to the client application.
- The customer information is then stored in the cache using Distributed data structures such as Lists. This will help in making decisions on future transactions much easier.
- The data is persisted in the database using Write-behind.
Managing Transactions with Pub/Sub
The Transaction Request Manager is initialized with the cache. Transaction Request Manager creates the transaction as a message and publishes it to its topic.
|
1 2 3 4 5 |
public void CreateRelevantSubscription() { MessageReceivedCallback transactionmessageReceivedCallback = new TransactionCompletedMessage(this).MessageReceivedCallback; transactionSubscription = base.CreateRelevantSubscriptions(Topics.REPLIESTOPICS, transactionmessageReceivedCallback); } |
Whenever a transaction is made through the Transaction Request Manager, it first creates a message from the customer information and then publishes this as a message on the transaction topic. This topic is subscribed by the Fraud Detection Manager, which receives the message and performs fraud detection logic on the incoming message and sends the results back to the Transaction Request Manager.
|
1 2 3 4 5 |
// Sending messages on the respective topic if (customer != null) { ncache.PublishMessageOnTopic(Topics.TRANSACTIONTOPICS, CreateTransactionFromCustomer(customer), null); } |
Detecting Fraud in Real-Time
The Fraud Detector acts as an Event-Driven handler, performing Real-Time Anomaly Detection by comparing incoming transaction patterns against historical benchmarks stored in-memory. It manages all verification logic and maintains a persistent, low-latency connection with the NCache cluster. The Fraud Detection Manager is initialized by creating a topic in NCache and subscribing to this topic. The callback for the Fraud Detection Manager topic exists in the Transaction Request Manager logic as it sends the results of the fraud detection logic back to the Transaction Request Manager.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
public void InitiliazeFraudManager() { base.CreateRelaventTopics(Topics.REPLIESTOPICS); base.CreateRelaventTopics(Topics.TRANSACTIONTOPICS); CreateRelevantSubscription(); } public void CreateRelevantSubscription() { // Initializes the relevant subscribers with their callbacks MessageReceivedCallback transacionManagermessageReceivedCallback = new StartTransactionMessage(this).MessageReceivedCallback; transactionSubscription = base.CreateRelevantSubscriptions(Topics.TRANSACTIONTOPICS, transacionManagermessageReceivedCallback); } |
During the execution of the fraud detection logic, the system follows a high-density verification sequence:
- State Verification: Customer data is retrieved from the in-memory cache
- Historical Check: The system verifies if the last cached transaction was flagged as fraudulent. If so, the current transaction is failed immediately
- Pattern Analysis: If no immediate red flags exist, the system applies learned logic to the current data
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
if (customerInfo != null) { if (learntLogic.FraudFoundInLastTransactions(transactionMessage, customerInfo)) { result = Result.Faliure; } } if (result != Result.Faliure) { bool isValid = learntLogic.IsTransactionValid(transactionMessage); if (!isValid) result = Result.Suspicious; } |
In this learned logic, we use multiple factors to determine the validity of the transaction. We check if a transaction has originated from a valid IPAddress and a valid email address that lies in a proper domain. Moreover, the email shouldn’t exist in a range of suspicious emails. It is also checked if the transaction originated from any suspicious location. If any of the above criteria fails, the failure weight is incremented. If the failure weight is greater than the decision weight, the transaction as invalid. You can find the code for this logic at GitHub.
Caching Results and Persist them to Database
Once the result has been declared, the result is added/updated in the cache.
|
1 2 |
FraudRequest fraudRequest = CreateFraudRequest(transactionMessage, result); ncache.UpdateCustomerInfoInCache(customerkey, fraudRequest); |
The cache is configured for the data source. For storing customer information in the in-memory cluster, we have used NCache’s distributed lists, which are one of the distributed data structures. The information of the customers is also persisted to the database using Write-Behind, which asynchronously updates the customer information.
|
1 2 3 4 5 6 7 8 9 10 11 |
IDistributedList list = cache.DataTypeManager.GetList(key); if (list == null) AddCustomerInCache(key, null, cutomerInfo); else { // Update info of a customer against its id list.WriteThruOptions = new WriteThruOptions(WriteMode.WriteBehind); list.Add(cutomerInfo); } |
Once cached, a new message is created from this result and published back on the topic subscribed by the Transaction Request Manager. It receives the message and publishes the result back to the client application.
Conclusion
NCache is a distributed in-memory .NET cache that comes with multiple features like Pub/Sub, distributed data structures, data source providers and more. By offering a native .NET stack, NCache removes the serialization overhead found in other caching solutions, making it the ideal choice for high-stakes fraud detection where every millisecond counts. The purpose of storing and processing data in local memory is eliminating the cost of network transfer, as all processing is done through the cache, and results are later updated on the database.
Frequently Asked Questions (FAQ)
Q: How does the Fraud Detection Manager identify suspicious transactions?
A: It calculates a cumulative “failure weight” by checking if the transaction originates from a valid IP address, a trusted email domain, or a known geographical location. If the total weight exceeds a specific threshold, the transaction is flagged.
Q: What happens if a transaction is flagged as “fraudulent” in a previous session?
A: The system performs a historical state check by retrieving the customer’s record from the in-memory cache. If the most recent transaction was fraudulent, the system fails the current attempt immediately before running further logic.
Q: Why is NCache used instead of a standard relational database for this logic?
A: Standard databases often require disk I/O, which is too slow for real-time counters. NCache provides “blazing fast” results by keeping data in local memory, which is essential to prevent customers from waiting at the point of sale.
Q: How is the machine learning model improved over time?
A: The system uses an Offline-Processing System. This layer takes data from the live environment to improve patterns and retrain ML algorithms, then feeds the upgraded logic back into the Near-Real-Time (NRT) system.
Q: How does the system handle transaction results once a decision is made?
A: Once the Fraud Detector declares a result, it updates the customer’s information in an NCache Distributed List. A message is then published back to the Transaction Request Manager via a Pub/Sub topic to notify the client application.
Enjoyed reading the article above, really explains everything in detail, the article is very interesting and effective. Thank you and good luck with the upcoming articles.