Score
Design and engineering of algorithms and strategies that adapt interventions, recommendations, or policies to individual users (using RL, engineering constraints, or federated pipelines), including mechanisms for rapid iteration, integration, and individualized featureization.
The application of reinforcement learning (RL) in software engineering lacks a systematic, comprehensive survey. Method: We conduct the first panoramic systematic literature review (SLR), covering 22 top-tier conferences and synthesizing 115 deep RL studies. Using multidimensional analysis—spanning algorithm types, datasets, model architectures, and evaluation methodologies—we develop a taxonomy organized along four software engineering activities: design, development, quality assurance, and maintenance. Contribution/Results: Our analysis reveals prevailing practices, shared limitations—including data scarcity, inconsistent evaluation protocols, and poor reproducibility—and emerging evolutionary trends. We propose actionable recommendations to address these challenges and publicly release all research artifacts, including the curated literature corpus, coding scheme, and analysis tools. This open resource establishes an extensible benchmarking framework and identifies concrete directions for future research in RL for software engineering.
This study addresses the prevailing gap in AI education, which emphasizes model development while neglecting system engineering practices, leaving students ill-equipped to handle real-world challenges such as architectural design, deployment, and monitoring. To bridge this gap, the authors implemented a master’s-level course in which students built a movie recommendation system under realistic constraints, with a focus on integrating AI components into robust software systems, adopting data-driven machine learning practices, and cultivating systems-level thinking. Using a mixed-methods approach—combining analysis of student project artifacts with survey data—the research evaluates learners’ performance in architectural decision-making, integration of heterogeneous models, and adaptation to evolving requirements. Findings reveal common difficulties students encounter in AI system engineering and demonstrate the course’s effectiveness in addressing critical deficiencies in AI engineering education and enhancing systems-aware competencies.
This work addresses the lack of a precise definition of “personalization” in existing algorithmic recourse methods, which hinders systematic evaluation of its impact on effectiveness, cost, and reasonableness. The paper formalizes personalization as individualized actionability by incorporating hard constraints—restricting the set of actionable features—and soft constraints—modeling users’ preferences over the value and cost of recommended actions—within a causal recourse framework. It further introduces a pre-recourse user prompting mechanism to enable personalized recommendations. Experimental results demonstrate that hard constraints substantially reduce both the effectiveness and reasonableness of recourse suggestions. Moreover, significant disparities emerge across social groups in terms of recourse cost and reasonableness, revealing a complex trade-off between personalized design and fairness.
In response to emergent risks posed by advanced AI—including election interference, cyberterrorism, and systemic failure—this paper proposes a novel risk governance paradigm centered on “societal adaptability,” moving beyond conventional technical containment toward systemic resilience. Methodologically, it establishes a dynamic, three-phase “avoid–defend–recover” cycle, integrating conceptual modeling, multi-scenario risk analysis, policy intervention design, and a cross-stakeholder governance framework involving governments, industry, and independent third parties. Key contributions include: (1) the first formal theoretical framework for societal adaptability in AI governance; (2) an actionable, stage-wise adaptation pathway; and (3) a globally applicable, operational AI governance system that reconciles stringent safety safeguards with sustained innovation capacity. This work delivers an original, implementation-oriented solution for balancing AI risk mitigation and technological advancement. (149 words)
To address insufficient stakeholder engagement and the lack of iterative validation in large language model (LLM) alignment, this paper introduces the “policy prototyping” paradigm—a human-centered, collaborative framework for designing LLM behavioral policies. Methodologically, it integrates human-AI co-design, rapid policy sandbox experimentation, multi-round cross-stakeholder workshops, and an empirically grounded iterative evaluation framework—replacing traditional linear alignment with a closed-loop “intention–feedback–revision” cycle. Key contributions include: (1) establishing the first principled foundation for policy prototyping; (2) ensuring fidelity between collective stakeholder input and actual model behavior; and (3) demonstrating in an industrial AI lab that the approach significantly improves policy interpretability, intention fidelity, and cross-group consensus—thereby extending the methodological frontier of collaborative alignment. (149 words)
This study addresses the limited interpretability and clinical credibility of reinforcement learning (RL) decisions in dynamic treatment regimes (DTRs). To this end, we propose a medical-knowledge-driven interpretable RL framework. Methodologically, we introduce, for the first time, a mathematical coupling mechanism between medical prior knowledge and RL policy learning—leveraging expert-curated knowledge graphs to guide state representation, incorporating clinical rules into reward shaping, integrating domain constraints into Markov decision process (MDP)-based deep Q-networks, and employing counterfactual policy evaluation for robust assessment. This enables structured, safety-aware constraint and guidance of the policy space by domain knowledge. Evaluated on both synthetic simulations and real-world electronic health record data, our approach reduces adverse event incidence by 37% and improves clinical consistency (as measured by expert scoring) by 42%. The work establishes a novel paradigm for DTR deployment that jointly ensures safety, interpretability, and personalization.
This work addresses the lack of systematic methodologies in model optimization, which often relies on heuristic choices and struggles to accommodate diverse deployment constraints. It formalizes model compression and acceleration as a constraint-aware multi-objective engineering decision problem, establishing a unified and actionable framework grounded in five key dimensions: data availability, latency, memory footprint, accuracy tolerance, and retraining budget. By integrating techniques such as quantization, pruning, knowledge distillation, parameter-efficient fine-tuning (PEFT), and inference optimization, the study proposes tailored optimization pipelines for four representative industrial scenarios, delivering a reproducible and quantifiable guide for technology selection.
Current AI agents predominantly rely on ad hoc, on-the-fly generation strategies and lack the reliability safeguards inherent in established software engineering practices, rendering them ill-suited for high-stakes scenarios demanding stringent safety and robustness. This work proposes a novel “AI workflow store” paradigm that systematically integrates software engineering principles—such as iterative design, rigorous testing, and adversarial evaluation—into AI agent architectures. By encapsulating reusable workflows, enforcing deterministic execution constraints, and adopting phased deployment strategies, the framework constructs a hardened library of high-assurance AI workflows. Empirical results demonstrate that this approach significantly outperforms conventional just-in-time synthesis methods in mission-critical tasks, achieving markedly enhanced safety and robustness without sacrificing flexibility.
This work proposes the first industrial-scale, end-to-end autonomous iteration framework for recommender systems, addressing the longstanding reliance on manual hypothesis formulation, handcrafted implementation, and labor-intensive experimental attribution that hinders scalable self-evolution. The framework employs a multi-agent collaborative architecture to automatically conceive, generate, validate across multiple dimensions, and deploy recommendation algorithms through online A/B testing. Central to this approach is the Semantic Gradient Policy Optimization (SGPO) mechanism, which enables continuous self-improvement by converting both successful and failed experiments into structured knowledge. Evaluated in real-world production environments, the system substantially increases experimental throughput and iteration velocity while progressively enhancing agent capabilities, thereby overcoming the fundamental bottlenecks of human-driven development.
This work addresses the challenge of rapidly adapting deployed operations research optimization models to new constraints or disturbances in dynamic real-world environments, where current approaches heavily rely on expert intervention. We propose a novel large language model (LLM)-based agent framework that embeds an LLM as an operations research expert within the reoptimization pipeline. The framework leverages natural language interaction to interpret evolving requirements, automatically generates structured model patches, and incorporates information from prior solutions to design acceleration strategies, enabling efficient and interpretable continuous adjustment. By integrating valid inequalities, solver tuning, and metaheuristics, the method demonstrates strong empirical performance in both online supply chain reoptimization and offline university exam timetabling, significantly improving computational efficiency while preserving solution quality and enabling rapid response with minimal expert dependency.
This study addresses the disruptive impact of large language models and AI agent systems—capable of generating vast volumes of code—on traditional software engineering paradigms. The work proposes a new paradigm centered on agent orchestration, verification of AI-generated code, and structured human-AI collaboration. Through a structured synthesis of literature review and industry practices, it constructs a comprehensive framework encompassing education, toolchains, lifecycle management, and governance. The research reveals a fundamental shift in the nature of code—from a scarce craft artifact to a consumable commodity—and identifies the evolving role of software engineers toward system design, semantic validation, and accountability oversight. It further establishes key directions such as a verification-first software development lifecycle, offering both theoretical grounding and practical pathways for software engineering transformation in the AI era.