proof of concept

Building a focused, minimal implementation to validate key technical or business assumptions — typically a short-lived prototype or experiment demonstrating feasibility, performance metrics, integration points, and risk areas before full investment.

proofofconcept

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

What's Coming Next? Short-Term Simulation of Business Processes from Current State

Sep 09, 2025
MA
Maksym Avramenko
🏛️ University of Tartu

Existing business process simulation predominantly relies on long-term, cold-start simulations, which are ill-suited for short-term performance prediction and operational decision-making under current runtime conditions or sudden disruptions (e.g., demand surges, resource shortages). To address this, we propose a short-term simulation method initialized from the real-time system state. Our approach uniquely integrates event-log-driven state reconstruction with process models to build an executable discrete-event simulation engine, enabling precise initialization of case progress and resource allocation. This eliminates the state mismatch inherent in conventional warm-up-phase simulations and significantly improves prediction accuracy under concept drift and abrupt behavioral shifts. Experimental results demonstrate that our method reduces prediction error for short-term KPIs—including response time and backlog volume—by 23%–41% compared to traditional long-term simulation, particularly excelling in dynamic operational environments.

Addressing operational decision-making with temporary disruptions impactInitializing simulation using event logs of ongoing casesSimulating business processes from current state for short-term forecasts

Tracking the Moving Target: A Framework for Continuous Evaluation of LLM Test Generation in Industry

Apr 26, 2025
MA
Maider Azanza
🏛️ University of the Basque Country UPV/EHU | LKS Next

Large language models (LLMs) deployed for industrial test generation face critical reliability challenges due to rapid model iteration, leading to outdated evaluations and compromised production trustworthiness. Method: This paper introduces the first continuous evaluation framework for LLM-based test generation tailored to industrial settings. It pioneers a “continuous evaluation” paradigm integrating technical metrics (e.g., code coverage) with engineering metrics (e.g., maintainability, expert ratings), while systematically addressing real-world issues including data leakage and irreproducible results. The framework integrates industrial toolchains (e.g., SonarQube), supports dynamic test-case selection, robust prompt engineering, and auditable measurement infrastructure. Contribution/Results: A longitudinal empirical study at LKS Next demonstrates that the framework accurately tracks LLM capability evolution, identifies key bottlenecks impeding industrial deployment, and effectively enables trustworthy integration into DevSecOps pipelines.

Addressing outdated effectiveness evaluations of rapidly evolving LLMsAssessing reliability and integration with DevSecOps practicesContinuous evaluation of LLM test generation in industry

This study addresses the long-standing lack of systematic measurement of “implementation risk” in quantitative investment backtesting—the performance discrepancies arising from differences in backtesting engine implementations. The work formally defines this risk for the first time and proposes four metrological metrics alongside a taxonomy of five failure modes. These are derived from parallel execution of 15 benchmark strategies across five open-source backtesting engines, incorporating transaction cost modeling, non-overlapping stratified asset buckets, and source code defect analysis. Experiments reveal that while engine outputs converge under zero-cost assumptions, performance divergence can reach up to 3.71% when transaction costs are introduced. Crucially, however, the relative ranking of strategy efficacy remains unchanged across engines (conclusion stability index = 1), indicating that implementation risk affects performance attribution but does not alter investment decisions.

backtesting enginesimplementation riskperformance divergence

Towards Lean Research Inception: Assessing Practical Relevance of Formulated Research Problems

Jun 15, 2025
AF
Anrafel Fernandes Pereira
🏛️ PUC-Rio | University of Bari | Blekinge Institute of Technology

Software engineering (SE) research frequently suffers from a misalignment between academic inquiry and industrial practice, particularly due to the absence of systematic evaluation mechanisms for *value*, *feasibility*, and *applicability*. To address this gap, we propose Lean Research Inception (LRI), the first framework to explicitly define and empirically validate these three interdependent criteria. LRI innovatively employs semantic differential scales to enable quantitative assessment and conducts empirical validation through industry–academia co-located workshops and joint evaluation sessions. Results from a multi-stakeholder study show that 83.3% of participants affirmed its value, 76.2% its feasibility, and 73.8% its applicability. Additionally, the evaluation yielded actionable insights—including terminology refinement and business-value articulation—that directly inform iterative framework improvement. LRI thus provides a practical, empirically grounded methodology to enhance the practice relevance and translational impact of SE research.

Aligning research value, feasibility, and applicability with industry needsAssessing practical relevance in Software Engineering research problemsEvaluating criteria for Lean Research Inception framework effectiveness

This study investigates how Danish software-focused small and medium-sized enterprises (SMEs) effectively demonstrate product security in B2B contexts to build client trust and meet regulatory compliance requirements. Method: Drawing on 16 semi-structured interviews and 6 participatory validation workshops, the research systematically identifies five security assurance practices: formal certifications, third-party audit reports, client questionnaires, interactive security meetings, and social proof. Contribution/Results: It introduces the first “Five-Dimensional Contextual Security Assurance Paradigm,” empirically revealing trade-offs among cost, credibility, timeliness, and customization across these practices—thereby refuting the assumption of a universal optimal solution. The study proposes a dynamic portfolio strategy framework that balances feasibility and effectiveness, enabling SMEs to adopt low-cost, context-adapted compliance pathways. Findings have been empirically validated and adopted by Danish industry stakeholders.

Demonstrating secure software development for SMEsExploring five security demonstration approaches for SMEsMeeting security criteria in business-to-business models

Latest Papers

What's happening recently
View more

This work addresses the protracted development cycles in traditional visual analytics (VA) prototyping that hinder rapid validation of novel ideas. The authors propose a scaffolded, AI-assisted development paradigm centered on the Artifact–Transform Workflow Language (ATWL) as a structured framework, integrating large language model–driven AI assistants with targeted expert interventions to efficiently construct high-quality VA prototypes within hours. The approach successfully instantiated innovative visual designs such as “soft Pareto fronts” and “constellation” groupings. Controlled experiments further revealed the critical influence of scaffolding design, timing of human-AI collaboration, and methods of knowledge injection on prototype quality, leading the authors to advocate for a taxonomy of knowledge expression in human-AI collaborative systems.

AI-assisted designhuman-AI collaborationrapid prototyping

This work proposes a systematic approach to derive task effectiveness requirements in the absence of explicit user needs. The method deconstructs task intent into context, functionality, constraints, critical dimensions, performance attributes, and architectural solutions, and introduces a task complexity factor to quantify the impact of external challenges and technology maturity. By integrating Best-Worst Scaling, it prioritizes critical dimensions based on stakeholder judgments. Through task decomposition modeling and quantitative complexity analysis, the framework supports integration with UAF/SysML artifacts and establishes a traceable mechanism for generating Tier 1 and Tier 2 requirements. The approach is validated using a close air support mission case study, effectively addressing a critical gap in requirements engineering when clear initial inputs are unavailable.

adaptive methodmission complexitymission effectiveness

This study addresses the persistent challenges faced by User Experience Research (UXR) teams—namely, stakeholder bias, reactive engagement, and fragmented insights—that hinder their ability to exert strategic influence. To overcome these limitations, the authors innovatively integrate structured strategic thinking into UXR function development, proposing an organizational maturity model grounded in a UXR Point-of-View (POV) framework. Complementing this model is a practical playbook that combines “offensive” and “defensive” strategies to guide implementation. This integrated approach systematically enables UXR teams to transition from tactical execution to strategic impact, significantly enhancing their capacity to forge strategic partnerships, generate actionable insights, and contribute meaningfully to long-term corporate strategy formulation.

institutional barriersresearch function maturitystakeholder bias

This study investigates whether the self-repair capability of frozen small code models in non-retrainable settings stems from repeated exposure to failed code or relies on external executable falsification feedback. To address this, we introduce a falsifiable methodology comprising feedback decomposition, content-controlled placebo design, matched-generation-budget control experiments, and executable auditing. We conduct large-scale evaluations on HumanEval+ and MBPP+ benchmarks using frozen models ranging from 0.5B to 1.5B parameters. Results show that blind resampling solves 18 more tasks than naive retrying; significant repair efficacy occurs only when feedback includes executable counterexamples, whereas pure instructions or content-irrelevant placebos yield no measurable improvement. These findings demonstrate that effective self-repair depends critically on external falsifying information rather than mere self-restatement.

falsificationfeedback decompositionfrozen code models

Hot Scholars

FA

Faruk Alpay

Computer Engineering, Bahçeşehir University
Artificial IntelligenceSymbolic ComputationRecursive SystemsAlpay Algebra
JS

Jeffrey Shallit

Professor Emeritus of Computer Science, University of Waterloo
automata theorycombinatorics on wordsnumber theoryalgebra
EE

Edith Elkind

Northwestern University
Algorithmic game theorycomputational social choiceartificial intelligence
AK

Andrey Kupavskii

Moscow Institute of Physics and Technology
combinatoricsdiscrete geometry