•  
  •  
 

Keywords

Agentic AI, Fraud detection, Retrieval-augmented generation, Large language models, Explainable AI, Human-in-the-loop

Document Type

Article

Abstract

As digital payment systems facilitate billions of transactions every day, there is a high probability of fraudulent activities in these systems. Traditional fraud detection systems, including rule-based systems and machine learning-based systems, have three major limitations: lack of explainability in terms of regulatory requirements, lack of contextual reasoning in terms of rare behavioral patterns, and lack of interaction with human experts in fraud analysis. This paper proposes a novel agentic framework in fraud detection systems by integrating Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) in context-aware reasoning based on historical transaction evidence. The framework is designed as a system of specialized agents working together in a coordinated manner in semantic retrieval, context-aware fraud analysis, and explanation generation. The vector-based retrieval system is designed to analyze transactions in behavioral context while minimizing hallucination risk associated with language models. The system is tested using a set of synthetic yet behaviorally realistic financial transactions that mimic class imbalance in real-world financial systems. The experimental results show better performance in terms of Precision = 0.985, Recall = 0.955, F1-score = 0.970, and AUCPR = 0.989 compared to traditional machine learning-based systems. The system is capable of generating structured explanations in a way that is useful for financial fraud analysis by humans. The results clearly show the effectiveness of retrieval-based agentic reasoning in financial fraud detection systems.

DOI

10.65645/3105-9104.1023

Download

Share

COinS