Problem
Research questions and friction points this paper is trying to address.
open-world software
software testing
non-determinism
uncertainty
test oracles
Software Testing Beyond Closed Worlds: Open-World Games as an Extreme Case
🤖 AI Summary
Traditional software testing relies on the closed-world assumption—such as finite state spaces and stable oracles—and struggles to address the uncertainty, non-determinism, and dynamic evolution inherent in modern systems. Taking open-world games as an extreme yet illustrative case, this work systematically identifies core testing challenges, including infinite behavioral spaces and unstable oracles, and proposes a novel testing paradigm that transcends the closed-world assumption. By integrating software testing theory, behavioral modeling, and uncertainty analysis, we develop a framework for representing and interpreting system behavior in uncertain environments. This framework clarifies key research directions, including automated test generation, evaluation metrics, and empirical study design, thereby establishing a theoretical foundation for testing complex, dynamic systems.
📝 Abstract
Software testing research has traditionally relied on closed-world assumptions, such as finite state spaces, reproducible executions, and stable test oracles. However, many modern software systems operate under uncertainty, non-determinism, and evolving conditions, challenging these assumptions. This paper uses open-world games as an extreme case to examine the limitations of closed-world testing. Through a set of observations grounded in prior work, we identify recurring characteristics that complicate testing in such systems, including inexhaustible behavior spaces, non-deterministic execution outcomes, elusive behavioral boundaries, and unstable test oracles. Based on these observations, we articulate a vision of software testing beyond closed-world assumptions, in which testing supports the characterization and interpretation of system behavior under uncertainty. We further discuss research directions for automated test generation, evaluation metrics, and empirical study design. Although open-world games serve as the motivating domain, the challenges and directions discussed in this paper extend to a broader class of software systems operating in dynamic and uncertain environments.
Innovation
Methods, ideas, or system contributions that make the work stand out.
open-world software
non-deterministic testing
uncertain environments
test oracles
behavior characterization
🔎 Similar Papers
No similar papers found.
