llm tooling

To address the challenge non-technical users face in constructing and deploying LLM-based agents, this paper introduces the first zero-code, self-evolving Agent Operating System. The system enables end-to-end generation of executable agents directly from natural language—requiring no programming expertise or manual intervention. Its core components include an LLM-driven executable engine, a self-managing file system, and a self-play-based customization module, integrated with multi-agent coordination, retrieval-augmented generation (RAG), autonomous tool synthesis, and natural-language-to-workflow compilation. Evaluated on the GAIA benchmark, our approach significantly outperforms existing state-of-the-art methods, achieving leading performance in both general multi-agent task solving and RAG-specific accuracy. This work establishes a novel paradigm for low-barrier, production-ready LLM agent development.

To address the challenge of adapting large language models (LLMs) to proprietary industrial programming languages—such as ABB RAPID—in automation domains, this paper proposes a fine-tuning-free, few-shot prompting method enabling locally deployed LLMs to directly comprehend and modify RAPID programs. By eliminating reliance on large-scale annotated datasets or custom model training, the approach preserves data privacy and enhances deployment flexibility. Experimental evaluation demonstrates its effectiveness on elementary tasks including code repair and logical adaptation, substantially lowering the adoption barrier for LLMs in non-general-purpose industrial language settings. Key contributions include: (i) the first systematic investigation into LLM support for closed industrial languages like RAPID; (ii) a lightweight, secure, and plug-and-play prompting framework; and (iii) a low-overhead, highly controllable paradigm for AI-assisted programming tailored to high-sensitivity industrial environments.

To address high redundant computation, large message latency, and slow end-to-end execution in multi-agent large language model (LLM) workflows, this paper proposes Prompt Choreography—a novel framework centered on the first dynamic global KV cache mechanism. This mechanism enables cross-agent and cross-invocation reuse of message encodings and attention redirection. Integrated with cache-aware fine-tuning and message-level attention subset selection, it preserves semantic consistency. Additionally, a parallel LLM invocation scheduling strategy is designed to maximize throughput. Experiments demonstrate a 2.0–6.2× reduction in first-token latency and over 2.2× speedup in end-to-end workflow execution—particularly pronounced under high-redundancy conditions. This work pioneers the application of dynamic KV caching to multi-agent collaborative reasoning, establishing a new paradigm for efficient LLM-based workflow orchestration.

This study evaluates the reliability of large language models (LLMs) in static security analysis of smart contracts, investigating whether they can replace or merely complement traditional tools. To this end, we introduce the first automated evaluation framework that systematically assesses LLM performance in vulnerability detection, quantitatively revealing— for the first time—high false positive rates stemming from lexical biases (e.g., identifier naming) and insufficient semantic validation. Through extensive experiments with diverse prompting strategies, we observe a pronounced trade-off between precision and recall. Our framework achieves 92% accuracy in classifying model outputs, demonstrating that current LLMs are ill-suited for standalone security auditing but show promise as collaborative aids to conventional static analysis tools, thereby underscoring the necessity of hybrid approaches.

Current LLM application development lacks systematic, practice-informed guidelines, leading to a growing gap between academic research and industrial engineering. Method: Drawing on transcribed texts from 189 real-world developer practice videos (2022–2024), we integrate BERTopic-based automated topic modeling with iterative human refinement to construct the first empirically grounded, production-oriented thematic map of LLM application development. Contribution/Results: The map identifies eight core themes—including design & architecture, model enhancement, infrastructure, and ethical risk—spanning 20 key issues. Design & Architecture emerges as the most densely populated theme, with RAG at its architectural center; prompt engineering, fine-tuning, deployment toolchains, and AI ethics are recurrent high-frequency concerns. Critically, the map exposes significant lags in academic research relative to industrial practice and delivers an actionable, empirically validated priority framework—thereby bridging a critical empirical gap in the LLM engineering knowledge base.

This work addresses the limitations of existing agent orchestration frameworks, which rely on external schedulers and incur substantial context overhead, require state-of-the-art large language models, and risk exposing proprietary workflows. To overcome these issues, the authors propose compiling multi-node agent workflows—comprising up to 55 nodes—directly into the weights of a small fine-tuned language model, thereby creating what they term “underground agents.” This approach provides the first systematic demonstration that complex workflows can be effectively internalized within model parameters. By integrating structured workflow representations, task-specific knowledge injection, and decision-hub modeling, the method achieves performance comparable to leading models on tasks such as travel booking, Zoom customer support, and insurance claims processing, while reducing inference costs by two orders of magnitude and substantially diminishing reliance on conventional orchestration frameworks.

This work addresses the fragmented collaboration between domain experts and developers in large language model (LLM) application development by proposing an end-to-end open-source platform that seamlessly integrates collaborative prompt editing, one-click batch experimentation, and human–AI hybrid evaluation for the first time. The platform supports batch scheduling across multiple models and prompts, real-time consistency metrics, version control, cost tracking, and result provenance. User studies demonstrate that the system significantly enhances cross-role collaboration efficiency, offers an intuitive interface, reduces time overhead, and has been successfully deployed in an online psychological counseling scenario, validating its effectiveness and practical utility.

This work addresses key challenges enterprises face when transitioning large language model (LLM) prototypes into production—namely, insufficient auditability, unpredictable behavior, and the absence of enforceable behavioral guarantees. The authors propose “harness engineering,” a methodology that restructures prompt-driven prototypes into auditable, traceable LLM agent architectures. For the first time, enterprise-grade behavioral contracts—including entity routing, source attribution, and output formatting—are formally encoded as executable code, establishing model-agnostic, verifiable safety boundaries. Through codified contracts, runtime validators, fault-injection testing, and model-swapping evaluations on data from 25 Korean publicly listed companies, the approach demonstrates 100% compliance with specified contracts across three hosted LLMs while preserving full functional utility (120/120), significantly outperforming pure prompting or external guardrail strategies.

Traditional requirements traceability approaches are labor-intensive, error-prone, and suffer from low precision, making it difficult to effectively establish links between requirements and software artifacts. This work proposes the first systematic prompt engineering framework tailored for requirements traceability, enhancing the zero-shot and few-shot performance of large language models (LLMs) through contextual role injection, integration of domain knowledge, and a label-aware strategy for selecting diverse exemplars. Evaluated on four cross-domain benchmark datasets, the proposed method achieves state-of-the-art F2 scores, significantly outperforming conventional information retrieval techniques, fine-tuned models, and existing LLM-based approaches. The results demonstrate its strong potential to support semi-automated traceability workflows in practical software engineering contexts.

This work addresses the resource constraints—such as memory consumption and inference latency—that often hinder large language models in multi-domain specialization. To overcome these limitations, the authors propose SkillWeave, a framework that decomposes general-purpose capabilities into lightweight, domain-specific modular units called skillpacks, complemented by SkillZip compression to produce highly efficient inference formats. This approach enables knowledge recombination, fine-grained fine-tuning, and low-latency multitask inference. Experimental results demonstrate that a 9B-parameter SkillWeave model outperforms multiple baselines—including a 32B monolithic model—on multitask and agent-based benchmarks, achieving up to 4× faster inference while significantly enhancing deployment efficiency across multiple domains under a fixed memory budget.