amazon bedrock

This paper addresses the systemic absence of responsible practices in foundational model development by introducing the first comprehensive, multimodal resource guide covering text, vision, and speech modalities. Through systematic literature review, cross-modal taxonomy construction, and tool-to-capability mapping, it identifies four critical structural gaps: (1) scarcity of multimodal and multilingual tooling; (2) weak capabilities in data curation and safety evaluation; (3) insufficient system-level monitoring and reproducibility infrastructure; and (4) lack of environmental impact assessment and release governance frameworks. The project delivers a curated practice inventory comprising 250+ open-source tools and resources spanning data governance, training optimization, safety auditing, carbon footprint analysis, and responsible deployment. Empirically grounded, the findings inform policy formulation, tool development, and standardization efforts—advancing AI development from heuristic practice toward a verifiable, auditable, and sustainable engineering paradigm.

This work addresses two critical challenges in Ethereum infrastructure: the limited elasticity and operational complexity of self-hosted nodes, and the lack of protocol-layer observability in managed blockchain services. To resolve these, we propose a hybrid cloud architecture built atop AWS Managed Blockchain, integrating a custom EC2-based observation node instrumented with Web3.py and JSON-RPC interfaces. Leveraging AWS IAM least-privilege policies and CDK-driven infrastructure-as-code deployment, our approach enables fine-grained, protocol-level monitoring. It is the first solution to support real-time collection of over 1,000 metrics—including mempool dynamics, transaction latency, and gas consumption—within a managed blockchain environment, with visualization via Amazon CloudWatch. The architecture balances cloud elasticity, security compliance, and protocol transparency, facilitating reproducible academic research and enterprise-grade on-chain monitoring. This work establishes a novel paradigm for observable, cloud-native decentralized infrastructure deployment.

MCP server development suffers from labor-intensive glue code and repetitive manual efforts in authentication and configuration. Method: This paper introduces AutoMCP, the first end-to-end compiler that automatically generates fully functional MCP servers directly from OpenAPI 2.0/3.0 specifications. It supports dynamic tool discovery, automatic schema registration, injection of authentication logic, and semantic interface mapping—eliminating the need for hand-crafted adaptation layers. Contribution/Results: Evaluated on 50 real-world APIs (>5,000 endpoints), AutoMCP achieves an initial call success rate of 76.5%, improving to 99.9% after lightweight fixes. The resulting corpus comprises 5,066 standardized, callable tools. This work substantially lowers the barrier to integrating external tools into LLM agents and advances scalability and protocol standardization within the MCP ecosystem.

Existing benchmarks evaluate single tools in isolation, overlooking real-world challenges such as functional overlap and cross-server orchestration—leading to distorted evaluations. This paper introduces MSC-Bench, the first end-to-end, multi-hop tool-coordination benchmark designed for the hierarchical Model-Context Protocol (MCP) ecosystem. Its contributions are threefold: (1) It constructs reproducible ground truth via “functionally equivalent sets,” enabling objective metrics (e.g., F1-score) and reducing reliance on LLM-based evaluation; (2) It proposes a five-level curriculum-style difficulty taxonomy to systematically diagnose failures in cross-server planning and out-of-scope request handling; (3) It comprehensively covers capabilities from single-tool invocation to complex, multi-server coordination. Experiments expose critical performance bottlenecks of current LLM agents under rigid hierarchical constraints. MSC-Bench is open-sourced, accompanied by a fine-grained attribution analysis framework to support rigorous agent evaluation and improvement.

Existing foundation model evaluations predominantly focus on isolated dimensions—such as benchmark accuracy or narrow task performance—lacking a holistic paradigm that integrates real-world application contexts, ethical implications, and engineering practicality. Method: This paper introduces the first structured, interdisciplinary evaluation science framework unifying use-case contextualization, ethical risk assessment, and systems engineering principles. It comprises a formalized evaluation methodology, an open-source toolkit (including standardized checklists and modular templates), and a systematic survey of state-of-the-art advances. Contribution/Results: The work pioneers a paradigm shift from fragmented, ad hoc evaluations to end-to-end, reproducible, and auditable assessment practices. Its methodology is fully open-sourced, enabling both industry and academia to conduct scalable, scenario-specific, and responsible foundation model evaluations grounded in rigorous scientific and ethical standards.

This study addresses the task of receipt item classification by systematically evaluating the trade-offs among accuracy, response stability, and inference cost for four leading commercial large language models—Claude 3.7 Sonnet, Claude 4 Sonnet, Mixtral 8x7B Instruct, and Mistral 7B Instruct—deployed on AWS Bedrock in a production environment. For the first time within a real-world, production-oriented framework, it conducts a multidimensional comparative assessment of zero-shot and few-shot prompting strategies. The results demonstrate that Claude 3.7 Sonnet achieves the optimal balance between classification performance and token-based inference cost, offering a cost-effective model selection rationale for practical deployment. Furthermore, the study highlights the critical influence of prompting strategy on overall cost-efficiency, underscoring its importance in operational decision-making.

This work addresses the absence of continuous governance mechanisms for dynamically balancing quality, reliability, safety, latency, and cost when enterprises deploy large language model (LLM) agents. The authors propose an Evaluation-Driven Development and Operations (EDDOps) framework tailored for non-deterministic LLM agents, built upon AWS AgentCore. By integrating observability, pluggable evaluators, and an agent registry, the architecture enables evidence-driven model selection and governance throughout the agent lifecycle. This approach reframes model selection from static benchmark rankings to an economic decision grounded in cost–performance trade-offs. The efficacy of the framework is demonstrated through 30 single-turn invocations, nine multi-turn evaluations, and successful integration with a registry, collectively validating its feasibility for efficient EDDOps implementation on managed platforms.

This work addresses the limitations of traditional workflow platforms, which rely on static, pre-defined processes and struggle to accommodate the dynamic data integration demands of distributed systems. To overcome this, the authors propose a configuration-driven runtime orchestration framework that dynamically constructs execution graphs at request time through dependency-aware scheduling and parallel task execution, thereby circumventing the constraints of fixed workflows. This approach enables rapid adaptation to evolving integration scenarios without requiring system redeployment, significantly reducing latency. Empirical evaluation in a real-world Customer 360 enterprise use case demonstrates that the framework offers substantial advantages in flexibility, scalability, and efficient data aggregation compared to conventional solutions.

This work addresses the lack of structured, verifiable, and governable tool support for large language model (LLM) agents in operational tasks, where existing approaches are often static or manually integrated, struggling to balance security and extensibility. The authors propose the “Tool Capsule” paradigm, which encapsulates tools as standardized units comprising intent, contract, implementation, policy, and verification evidence. They design an efficient intent-scoped routing mechanism enabling on-demand, secure tool invocation. The system integrates a sandboxed verification pipeline, MCP-compatible routing, credential binding, and lifecycle governance. Experiments demonstrate a micro F1 score of 0.901 across 83 routing tests with a 99.2% reduction in context overhead; all 25 end-to-end tasks produced valid toolkits (micro F1 = 0.940), with 23 successfully passing real-time sandbox validation.

This study addresses the evaluation of large language models’ capability to perform incremental modifications in Infrastructure-as-Code (IaC) development within cloud environments, specifically focusing on enterprise scenarios that require editing existing AWS CDK code based on natural language instructions. To this end, we introduce the first benchmark tailored for imperative IaC tools, leveraging real-world code repositories and natural-language-driven incremental editing tasks, with correctness validated through automated testing. Experimental results reveal that even state-of-the-art models exhibit limited performance—e.g., Sonnet 3.7 achieves only a 34% success rate—highlighting the significant challenges posed by reasoning over complex cloud resource dependencies and implementation patterns. This work fills a critical gap in existing benchmarks, which lack support for incremental editing in imperative IaC contexts.

This study addresses the unclear usage patterns and functional demands surrounding current MLOps frameworks in open-source projects, which hinder their effective evolution. For the first time, it systematically links real-world framework adoption with user enhancement requests by analyzing GitHub dependencies, API invocations, and issue reports across eight prominent MLOps frameworks, employing qualitative coding and thematic mapping. The findings reveal that developers prefer customized integrations over out-of-the-box solutions, and that these frameworks are seldom directly embedded in GitHub Workflows, instead being primarily applied to core machine learning phases and infrastructure governance. Users most frequently request enhancements to core functionality, greater API exposure, and improved CI/CD integration, while increasingly adopting multiple frameworks in tandem.