ScaleCall - Agentic Tool Calling at Scale for Fintech: Challenges, Methods, and Deployment Insights

2025-10-29

AI Summary

In highly regulated domains such as fintech, LLM-based tool invocation faces challenges including on-premises deployment requirements, stringent compliance constraints, and ambiguity arising from functional overlaps among APIs. To address these, this paper proposes ScaleCall—a production-ready framework for enterprise-grade API orchestration and automated data engineering. Its core innovation is a flexible hybrid tool retrieval architecture integrating embedding-based retrieval, prompt-driven ranking, and collaborative re-ranking. Empirical analysis demonstrates that domain-specific characteristics exert a stronger influence on retrieval performance than algorithmic choices alone. Evaluated in Mastercard’s real-world, regulated environment, embedding-based retrieval achieves low latency and superior ambiguity resolution in large-scale tool repositories, while the hybrid strategy shows promise in high-precision scenarios. This work establishes a deployable paradigm and a systematic evaluation benchmark for LLM tool invocation in compliance-sensitive settings.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

While Large Language Models (LLMs) excel at tool calling, deploying these capabilities in regulated enterprise environments such as fintech presents unique challenges due to on-premises constraints, regulatory compliance requirements, and the need to disambiguate large, functionally overlapping toolsets. In this paper, we present a comprehensive study of tool retrieval methods for enterprise environments through the development and deployment of ScaleCall, a prototype tool-calling framework within Mastercard designed for orchestrating internal APIs and automating data engineering workflows. We systematically evaluate embedding-based retrieval, prompt-based listwise ranking, and hybrid approaches, revealing that method effectiveness depends heavily on domain-specific factors rather than inherent algorithmic superiority. Through empirical investigation on enterprise-derived benchmarks, we find that embedding-based methods offer superior latency for large tool repositories, while listwise ranking provides better disambiguation for overlapping functionalities, with hybrid approaches showing promise in specific contexts. We integrate our findings into ScaleCall's flexible architecture and validate the framework through real-world deployment in Mastercard's regulated environment. Our work provides practical insights into the trade-offs between retrieval accuracy, computational efficiency, and operational requirements, contributing to the understanding of tool-calling system design for enterprise applications in regulated industries.

Problem

• Deploying LLM tool calling in regulated fintech environments
• Disambiguating large functionally overlapping toolsets for enterprises
• Balancing retrieval accuracy with computational efficiency requirements

Innovation

• Embedding-based retrieval for low-latency tool access
• Listwise ranking to disambiguate overlapping functionalities
• Hybrid architecture balancing accuracy and efficiency