training dynamics analysis and monitoring

In supervised cybersecurity CTF training, evaluating learning outcomes and identifying flaws in training design remain challenging. To address these issues, this paper proposes an evaluation framework integrating process mining with multidimensional visual analytics. It models participants’ operational behavior sequences and implements an open-source, interactive dashboard supporting temporal pattern recognition, multivariate network visualization, and clustering analysis—rigorously adhering to established visualization design principles. Our key innovation lies in deeply embedding process mining into cybersecurity pedagogical assessment, enabling automated discovery of process deviations, bottlenecks, and organizational anomalies directly from system logs. A case study demonstrates that the framework effectively quantifies learner engagement, pinpoints training deficiencies—including task bottlenecks and imbalanced resource allocation—and substantially enhances the interpretability of evaluation results and their utility for instructional improvement.

This work addresses the lack of a general, auditable dynamic control mechanism in existing training systems, which typically rely on framework-specific code. The authors propose the first cross-framework, open-source control plane that exposes training interfaces through a unified protocol, integrating declarative configuration, request validation, and secure control-point scheduling within the Aim workspace to enable metric monitoring, real-time intervention, and operational traceability. The system supports safe human and automated controller interventions during training while fully logging all operational trajectories. Experiments across five NLP and reinforcement learning tasks demonstrate its effectiveness, and the open-source implementation provides a foundation for reproducible human-in-the-loop training.

This study addresses the coupled challenges of class imbalance and class overlap in software defect prediction, which jointly impair model training dynamics and performance. The authors propose the first interaction-aware protocol for analyzing training dynamics under these intertwined data quality issues. By training a fixed multilayer perceptron (MLP) under three conditions—imbalance only, overlap only, and their coupling—the protocol systematically records training trajectories. Integrating effect size analysis, sensitivity analysis, and rule-based classification, it constructs a taxonomy of training dynamic patterns. The work uncovers distinctive neural network behaviors specific to the coupled scenario, offering empirical insights and novel diagnostic tools to enhance the understanding, evaluation, and refinement of defect prediction models.

Existing student performance prediction models are predominantly static and lack adaptability to post-intervention data, limiting their utility in dynamic educational settings. Method: We propose a feedback-driven, closed-loop self-optimizing decision support system that incrementally re-trains a LightGBM regression model using newly acquired post-intervention grades. The system features an interactive Flask-based web interface, integrates SHAP for feature attribution and model interpretability, and supports interoperability with Learning Management Systems (LMS) and institutional dashboards. Contribution/Results: By transforming static prediction into an online-evolving intelligent tutoring aid, the system enhances transparency and responsiveness in human-AI collaborative decision-making. Experimental evaluation demonstrates a 10.7% reduction in RMSE after incremental re-training and a consistent upward trend in predicted scores for intervened students, validating its effectiveness and adaptability in real-world educational environments.

Traditional neural network training relies on fixed optimization pipelines, rendering it inflexible in dynamically addressing training instability and anomalies. To address this limitation, we propose the first interactive training framework enabling real-time human–AI collaborative intervention. Our method employs a lightweight control server that integrates expert human directives with AI agent feedback to dynamically adjust hyperparameters, data sampling strategies, and model checkpoints during training. This framework introduces, for the first time, a closed-loop interactive paradigm into neural network training, establishing a scalable human–machine collaboration interface coupled with automated response mechanisms. Experimental results demonstrate significant improvements in training stability, reduced sensitivity to initial hyperparameter configurations, and enhanced real-time responsiveness to user-specified customization requirements. The effectiveness is validated across multiple benchmark tasks.

Current AI research often treats models as static artifacts, overlooking the fundamental influence of training dynamics on critical properties such as capability, bias, robustness, and safety. This work proposes shifting the focus toward the training process itself to establish a science of AI centered on training dynamics. By analyzing the interactions among data, objectives, architectures, and optimizers, the paper develops a theoretical framework that is predictive, intervenable, and design-oriented. Integrating approaches from mechanistic interpretability, fairness, memory mechanisms, and simplicity biases, it uncovers causal links between early-training signals and final model behavior. The study systematically outlines key challenges and open problems, offering both theoretical pathways and practical foundations for extending scaling laws beyond performance to encompass multidimensional model attributes.

This work addresses the vulnerability of neural network training to rare yet severe unstable updates, which can cause irreversible divergence or subtle performance degradation—issues that existing optimizers fail to detect or mitigate at runtime. The authors model the optimization process as a controlled stochastic process and introduce the first runtime stability framework that operates without modifying the underlying optimizer. By leveraging secondary signals such as validation probes, the framework automatically detects instability and triggers lightweight interventions grounded in control theory. Designed for memory-constrained settings, the approach offers low computational overhead, broad compatibility with standard optimizers, and theoretical guarantees of bounded degradation and recovery, effectively preventing training collapse and performance deterioration.

This work addresses the limitations of conventional training pipelines, which struggle to dynamically mitigate issues such as overfitting, loss imbalance, and unsafe exploration due to reliance on fixed policies or single-axis schedulers. The authors propose a large language model–based bounded supervisory controller that leverages structured telemetry snapshots to monitor training dynamics in real time and generates verifiable multi-parameter adjustment commands within a constrained action space. This enables closed-loop regulation of learning rate, regularization strength, loss weighting, and exploration strategy. Notably, it introduces pattern-constrained large language models into training supervision for the first time, supporting asynchronous, auditable multi-axis interventions applicable to both supervised and reinforcement learning. Experiments demonstrate a 60% loss reduction on TinyStories with effective overfitting correction, marked alleviation of overly conservative or unsafe exploration in robotic manipulation tasks, and generation of traceable intervention logs.

Access to real-world Applied Behavior Analysis (ABA) session data is severely limited by privacy constraints, hindering the training of AI models in this domain. To address this challenge, this work proposes a deterministic synthetic data generation method grounded in authoritative ABA taxonomies, enabling—for the first time—the construction of fully traceable instruction-tuning datasets. The approach supports two core tasks: instructional program generation and multi-session behavioral trajectory interpretation, while integrating standard ABA paradigms such as Discrete Trial Teaching and Natural Environment Teaching. The resulting TRACE dataset comprises 2,999 structurally transparent and content-compliant samples, partitioned into training, validation, test, and reasonableness-check splits according to predefined ratios. Both code and data are publicly released under CC BY-NC 4.0 and MIT licenses.

This work addresses the lack of fine-grained, quantitative assessment of learning processes and code quality in current programming education, which hinders accurate diagnosis of students’ comprehension and instructional efficacy. The authors propose a plugin system integrated into mainstream code editors that, for the first time, adapts industrial-grade development log analysis to educational contexts. By continuously capturing students’ coding behaviors, error messages, and progress data in real time, the system constructs a timestamp-driven behavioral tracking model to derive quantitative metrics. This approach enables structured recording and analysis of programming activities, facilitating timely evaluation of instructional content, identification of common learning bottlenecks, and the creation of an open programming behavior database tailored for educational research and personalized learning.