Institution profile

RMIT University

Academic institutionaustralasia · au
Official website
Research library425linked papers
Opportunities0open roles
Selected work

Representative Papers

Can Generative LLMs Create Query Variants for Test Collections? An Exploratory Study

Jul 18, 2023Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

This study investigates whether large language models (LLMs) can automatically generate high-quality search queries and their semantic variants from information need descriptions to enable low-cost, scalable test collection construction for information retrieval (IR). We propose a prompt-engineering-based LLM query generation method and design a multidimensional evaluation framework measuring semantic similarity, document pool coverage, and relevant document overlap. To our knowledge, this is the first systematic validation of LLM-generated variants for Chinese IR test collection construction. While LLM variants exhibit slightly lower diversity than human-annotated ones, they achieve a 71.1% relevant document overlap rate at pool depth 100—significantly outperforming baseline methods. Our results demonstrate that LLMs can effectively support automated, high-fidelity test collection creation, offering a novel, practical paradigm for IR evaluation infrastructure.

29 citationsRead paper

Understanding the Issues, Their Causes and Solutions in Microservices Systems: An Empirical Study

Feb 03, 2023arXiv.org

Microservice system developers lack empirical evidence regarding the types, root causes, and remediation strategies of recurring issues. Method: We adopt a mixed-methods approach—quantitatively analyzing 2,641 open-source issues, qualitatively interviewing 15 practitioners, and conducting a global survey with 150 practitioners. Contribution/Results: We introduce the first comprehensive, domain-specific three-level taxonomy (“Issue–Cause–Solution”) for microservices. We identify five high-frequency issue domains—including technical debt, CI/CD pipeline failures, and exception handling—and three predominant root causes, notably generic programming errors. From our analysis, we distill 177 actionable, context-aware remediation strategies. This work establishes an empirical foundation for microservice fault diagnosis and mitigation, delivers practical guidance for industry practitioners, and pinpoints critical research directions for next-generation microservice engineering.

6 citations2 influentialRead paper

MORTAR: Metamorphic Multi-turn Testing for LLM-based Dialogue Systems

Dec 20, 2024arXiv.org

To address the long-standing test oracle problem in multi-turn LLM-based dialogue systems, this paper proposes the first knowledge graph–driven metamorphic testing framework for multi-turn dialogues. Methodologically: (1) it constructs dialogue-level knowledge graphs to model semantic structures across turns; and (2) it designs dialogue-level perturbations and metamorphic relations to enable reference-free, LLM-free automated test case generation. The key contribution is the first systematic adaptation of metamorphic testing to multi-turn dialogue scenarios—circumventing evaluation bias introduced by LLM-based oracles. Experiments on multiple mainstream LLM dialogue systems demonstrate that our approach detects significantly more unique defects than existing methods; notably, its detection rate for severe defects is four times higher than that of the best-performing single-turn metamorphic testing baseline, while maintaining low computational cost and high reliability.

2 citationsRead paper

Performance-Driven QUBO for Recommender Systems on Quantum Annealers

Oct 20, 2024arXiv.org

To address the feature selection challenge in recommender systems, this paper proposes CAQUBO—a novel method that tightly integrates counterfactual analysis with Quadratic Unconstrained Binary Optimization (QUBO) modeling. CAQUBO quantifies the causal effects of individual and combined features on recommendation performance, constructing an interpretable QUBO coefficient matrix that guides D-Wave quantum annealers to efficiently identify the optimal feature subset. By establishing an explicit causal mapping from features to recommendation accuracy, CAQUBO overcomes the interpretability and performance limitations of conventional black-box, quantum-inspired heuristics. Extensive experiments across multiple benchmark datasets demonstrate that CAQUBO achieves an average 7.2% improvement in recommendation accuracy over existing quantum annealing–based feature selection methods, empirically validating the efficacy of the “quantum–causal synergy” paradigm.

1 citationsRead paper
Recent publications

Latest Papers