Policy Loopholes in Agent Evaluation: When Policy Ambiguity Masquerades as Agent Error
研究解决了自然语言策略中的模糊性导致的评估不准确问题，通过审计两个基准领域，提出了一种分类方法，并建议在收集标准注释前应先审核策略。
0 citationsRead paper
Selected work
Representative Papers
Reducing Per-Sample Harm in Stochastic Optimization
This work addresses the single-sample interference problem in stochastic optimization—where parameter updates inadvertently increase the loss of individual samples due to momentum or other historical state dependencies—by reformulating the update process as a constrained optimization problem that explicitly minimizes such interference. The key insight is that the final layer alone suffices to accurately capture the network-wide second-order statistical properties essential for this mitigation. Leveraging this observation, the authors construct a computationally efficient proxy problem whose dimensionality scales with batch size rather than model size, thereby reducing relative computational overhead as models grow larger. The proposed method is compatible with mainstream optimizers such as SGD and AdamW and employs a GPU-friendly iterative solver. Experiments demonstrate that it effectively suppresses single-sample interference and consistently improves generalization performance on standard image classification benchmarks.
0 citationsRead paper
Circula-based multivariate distributions on the flat torus, with applications in structural biology
This work addresses the challenge of modeling the joint distribution of backbone and side-chain torsion angles of adjacent amino acids in proteins on a flat torus. It proposes the first d-dimensional toroidal copula distribution featuring a closed-form normalizing constant and an explicit covariance structure. Built upon a low-rank latent variable model, the approach constructs high-dimensional toroidal mixture distributions and achieves scalable modeling from T² to T¹⁴ for the first time. The proposed model attains state-of-the-art performance in both likelihood and sparsity, establishing a new paradigm for statistical modeling of local protein conformations and advancing structural biology toward deeper integration with thermodynamic and kinetic analyses.
0 citationsRead paper
The Good, the Better, and the Best: Improving the Discriminability of Face Embeddings through Attribute-aware Learning
Existing face recognition methods often exhibit unstable performance under variations in age, pose, and occlusion, and typically rely on a fixed set of facial attributes for auxiliary supervision, overlooking the heterogeneous contributions of these attributes to identity discrimination. This work proposes an attribute-aware multi-task learning framework that explicitly groups attributes based on their relevance to identity, decoupling identity-relevant from identity-irrelevant attributes. By doing so, the model is guided to focus on identity-critical regions while suppressing irrelevant features. The approach not only leverages attribute learning as a diagnostic tool for shortcut learning but also significantly enhances the discriminability of learned embeddings on standard face verification benchmarks, thereby validating the effectiveness of selectively exploiting identity-relevant attributes and actively suppressing non-informative ones.
0 citationsRead paper
Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation
This work addresses a critical limitation in current agent evaluation paradigms, which focus solely on task completion while neglecting the compliance and consistency of execution processes, thereby misclassifying numerous “corrupted successes” as valid. To remedy this, we propose the Process-Aware Evaluation (PAE) framework, which models agent behavior as structured programs and introduces a multidimensional assessment across utility, efficiency, interaction quality, and process integrity, complemented by a gating mechanism to filter out non-compliant outcomes. Experiments on Tau-Bench reveal that 27%–78% of purported successes are in fact corrupted. PAE substantially reduces Pass⁴ pass rates, alters model rankings, and exposes structural flaws in existing benchmarks, thereby establishing— for the first time—a systematic, process-centric paradigm for agent evaluation.
0 citationsRead paper
Recent publications
Latest Papers
Policy Loopholes in Agent Evaluation: When Policy Ambiguity Masquerades as Agent Error
研究解决了自然语言策略中的模糊性导致的评估不准确问题，通过审计两个基准领域，提出了一种分类方法，并建议在收集标准注释前应先审核策略。
0 citationsRead paper
Reducing Per-Sample Harm in Stochastic Optimization
This work addresses the single-sample interference problem in stochastic optimization—where parameter updates inadvertently increase the loss of individual samples due to momentum or other historical state dependencies—by reformulating the update process as a constrained optimization problem that explicitly minimizes such interference. The key insight is that the final layer alone suffices to accurately capture the network-wide second-order statistical properties essential for this mitigation. Leveraging this observation, the authors construct a computationally efficient proxy problem whose dimensionality scales with batch size rather than model size, thereby reducing relative computational overhead as models grow larger. The proposed method is compatible with mainstream optimizers such as SGD and AdamW and employs a GPU-friendly iterative solver. Experiments demonstrate that it effectively suppresses single-sample interference and consistently improves generalization performance on standard image classification benchmarks.
0 citationsRead paper
Circula-based multivariate distributions on the flat torus, with applications in structural biology
This work addresses the challenge of modeling the joint distribution of backbone and side-chain torsion angles of adjacent amino acids in proteins on a flat torus. It proposes the first d-dimensional toroidal copula distribution featuring a closed-form normalizing constant and an explicit covariance structure. Built upon a low-rank latent variable model, the approach constructs high-dimensional toroidal mixture distributions and achieves scalable modeling from T² to T¹⁴ for the first time. The proposed model attains state-of-the-art performance in both likelihood and sparsity, establishing a new paradigm for statistical modeling of local protein conformations and advancing structural biology toward deeper integration with thermodynamic and kinetic analyses.
0 citationsRead paper
The Good, the Better, and the Best: Improving the Discriminability of Face Embeddings through Attribute-aware Learning
Existing face recognition methods often exhibit unstable performance under variations in age, pose, and occlusion, and typically rely on a fixed set of facial attributes for auxiliary supervision, overlooking the heterogeneous contributions of these attributes to identity discrimination. This work proposes an attribute-aware multi-task learning framework that explicitly groups attributes based on their relevance to identity, decoupling identity-relevant from identity-irrelevant attributes. By doing so, the model is guided to focus on identity-critical regions while suppressing irrelevant features. The approach not only leverages attribute learning as a diagnostic tool for shortcut learning but also significantly enhances the discriminability of learned embeddings on standard face verification benchmarks, thereby validating the effectiveness of selectively exploiting identity-relevant attributes and actively suppressing non-informative ones.
0 citationsRead paper
Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation
This work addresses a critical limitation in current agent evaluation paradigms, which focus solely on task completion while neglecting the compliance and consistency of execution processes, thereby misclassifying numerous “corrupted successes” as valid. To remedy this, we propose the Process-Aware Evaluation (PAE) framework, which models agent behavior as structured programs and introduces a multidimensional assessment across utility, efficiency, interaction quality, and process integrity, complemented by a gating mechanism to filter out non-compliant outcomes. Experiments on Tau-Bench reveal that 27%–78% of purported successes are in fact corrupted. PAE substantially reduces Pass⁴ pass rates, alters model rankings, and exposes structural flaws in existing benchmarks, thereby establishing— for the first time—a systematic, process-centric paradigm for agent evaluation.
0 citationsRead paper