Early Results from Teaching Modelling for Software Comprehension in New-Hire Onboarding

2025-10-08

AI Summary

New software engineers often struggle to comprehend large legacy systems, leading to prolonged onboarding periods. Method: This study introduces a systems-thinking training program grounded in Labelled Transition System (LTS) modeling and a structured understanding template—the first application of LTS modeling in software engineering onboarding education—featuring differentiated learning pathways across five sessions, integrating pedagogical best practices and pre-/post-assessment design. Contribution/Results: While overall comprehension gains were not statistically significant, learners with low initial proficiency showed a robust 15-percentage-point improvement (p < 0.05). Qualitative feedback indicated high engagement and perceived practical relevance. The framework offers a scalable, low-cost, reusable instructional model for cultivating software comprehension skills, addressing a critical gap in industry onboarding programs by introducing formal modeling techniques into foundational training.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Application Domains: Software Engineering
• Machine Learning: Hardware-aware ML

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Studies of user behavior, including longitudinal effects of personalized systems
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Working effectively with large, existing software systems requires strong comprehension skills, yet most graduates enter the industry with little preparation for this challenge. We report early results from a pilot intervention integrated into a SaaS company's onboarding program: a five-session course introducing systems thinking and Labelled Transition System (LTS) modelling. Participants articulated their understanding of product behaviour using a structured template and completed matched pre- and post-assessments. Of 35 new hires, 31 provided paired records for analysis. Across the full cohort, gains were small and not statistically significant. However, participants below the median on the pre-test improved by 15 percentage points on average (statistically significant), while those above the median regressed slightly (not statistically significant). Course feedback indicated high engagement and perceived applicability. These results suggest that short, modelling-focused onboarding interventions can accelerate comprehension for less-prepared new hires. At the same time, they point to the need for differentiated pathways for stronger participants, and to the potential for companies to adopt such interventions at scale as a low-cost complement to existing onboarding.

Problem

• Addressing software comprehension gaps in new-hire onboarding
• Evaluating modeling interventions for accelerating system understanding
• Providing scalable onboarding support for less-prepared graduates

Innovation

• Introduces Labelled Transition System modelling for software comprehension
• Uses structured templates to articulate product behavior understanding
• Implements short modeling-focused onboarding for less-prepared hires