Empirical Studies on Adversarial Reverse Engineering with Students

2026-03-04

AI Summary

This study addresses the persistent challenges in empirical reverse engineering research—namely, the scarcity of expert participants and the associated high costs—by systematically investigating the feasibility of using student populations as viable alternatives. The work proposes a methodological framework that balances scientific rigor with practical constraints, developed through a systematic literature review, an embedded empirical experiment within a master’s-level course, and mixed-methods data analysis. The resulting reproducible methodology for user studies in reverse engineering is accompanied by practical guidelines on privacy preservation, incentive design, and validity verification. This contribution not only demonstrates the potential of student-centric approaches but also establishes a foundational pathway for future empirical investigations in the field.

Technology Category

• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Application Domains: Humanities & Computational Social Science
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Empirical research in reverse engineering and software protection is crucial for evaluating the efficacy of methods designed to protect software against unauthorized access and tampering. However, conducting such studies with professional reverse engineers presents significant challenges, including access to professionals and affordability. This paper explores the use of students as participants in empirical reverse engineering experiments, examining their suitability and the necessary training; the design of appropriate challenges; strategies for ensuring the rigor and validity of the research and its results; ways to maintain students' privacy, motivation, and voluntary participation; and data collection methods. We present a systematic literature review of existing reverse engineering experiments and user studies, a discussion of related work from the broader domain of software engineering that applies to reverse engineering experiments, an extensive discussion of our own experience running experiments ourselves in the context of a master-level software hacking and protection course, and recommendations based on this experience. Our findings aim to guide future empirical studies in RE, balancing practical constraints with the need for meaningful, reproducible results.

Problem

• reverse engineering
• empirical study
• student participants
• software protection
• research validity

Innovation

• empirical study
• reverse engineering
• student participants
• software protection
• experimental design