Institution profile

Australian National University

Academic institutionaustralasia · au
Official website
Research library645linked papers
Opportunities0open roles
Selected work

Representative Papers

Reinforcement Learning in POMDP's via Direct Gradient Ascent

Jun 29, 2000International Conference on Machine Learning

This paper addresses the challenge of estimating policy gradients in partially observable Markov decision processes (POMDPs). We propose GPOMDP, a direct policy gradient estimation algorithm that operates on a single trajectory and involves only one tunable parameter. GPOMDP extends the REINFORCE framework to average-reward optimization in POMDPs without requiring a transition model or prior domain knowledge. Its gradient estimator admits an explicit bias–variance trade-off interpretation and can be integrated with conjugate gradient methods to accelerate convergence. We provide theoretical convergence guarantees for the algorithm under standard regularity conditions. Empirical evaluations demonstrate that GPOMDP robustly approximates the true policy gradient in complex POMDP environments and reliably converges to locally optimal policies maximizing the average reward.

117 citations4 influentialRead paper

Scaling Internal-State Policy-Gradient Methods for POMDPs

Dec 02, 2025

This work addresses the challenge of learning memory-based policies for infinite-horizon partially observable Markov decision processes (POMDPs). We propose a scalable internal-state policy gradient method. Our approach unifies model-based optimization and model-free simulation training within a single framework. Technically, we introduce a novel gradient estimator and internal-state update mechanism that jointly enhance policy representational capacity and training stability. Additionally, we incorporate computationally efficient state compression and variance reduction techniques. Evaluated on large-scale POMDP benchmarks—including noisy robot navigation and cooperative multi-agent tasks—our method substantially outperforms existing memory-augmented policy approaches. It achieves superior convergence speed, policy performance, and generalization across diverse partially observable sequential decision-making scenarios. The proposed solution thus provides a practical, scalable reinforcement learning framework for complex POMDPs.

45 citations8 influentialRead paper

A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal

Nov 05, 2022arXiv.org

This paper presents a systematic survey of deep learning–based facial image restoration, focusing on denoising, super-resolution, deblurring, and artifact removal. Addressing challenges such as strong facial structural priors and complex degradation modeling, we propose the first holistic taxonomy of methods, a unified evaluation framework, and an open-source benchmark repository encompassing 20+ state-of-the-art approaches—including fully reproducible implementations. Leveraging datasets like CelebA and FFHQ, we conduct comprehensive cross-method evaluations using PSNR, SSIM, and LPIPS metrics, integrating CNN/Transformer architectures, perceptual and adversarial losses, and multi-scale feature fusion strategies. Our empirical analysis reveals performance boundaries and task-specific suitability across methods. Key contributions include: (1) the first structured, principle-driven classification system for facial restoration; (2) the first open, end-to-end benchmark platform supporting full method reproduction; (3) a rigorous, large-scale empirical study; and (4) concrete research directions concerning network design, evaluation paradigms, and dataset construction.

40 citations1 influentialRead paper

Reinforcement Learning From State and Temporal Differences

Dec 09, 2025

This paper identifies a fundamental limitation of TD(λ): although it minimizes mean-squared error in state-value estimates, policy quality depends critically on the *relative ordering* of state values—not their absolute accuracy. Consequently, even when initialized with the optimal policy, TD(λ) may converge to suboptimal policies, as empirically demonstrated in two- and three-state MDPs and Backgammon. To address this, we propose a novel framework that directly optimizes *state-value ranking consistency*, introducing STD(λ)—the first algorithm to formulate temporal-difference learning as a pairwise ranking problem. STD(λ) constructs a differentiable ranking loss from binary comparisons and integrates principles from both TD learning and contrastive (differential) training. We prove that, in two-state systems, STD(λ) guarantees monotonic policy improvement and convergence to the optimal policy. Experiments confirm that STD(λ) significantly outperforms TD(λ) in both synthetic two-state MDPs and a modified Acrobot task, and further reveal that TD(λ)’s value misranking is the root cause of its policy degradation in Backgammon.

9 citationsRead paper

Blockchain-enabled decentralized privacy-preserving group purchasing for retail energy plans

Jun 28, 2022Energy-Efficient Computing and Networking

In traditional retail energy markets, group purchasing reliant on trusted third parties suffers from privacy leakage and insufficient transparency. This paper proposes a decentralized, privacy-preserving group purchasing mechanism that integrates Ethereum blockchain with secure multi-party computation (SMPC). Users collaboratively evaluate energy plans and jointly determine optimal supplier switching without revealing private consumption data. We design a publicly verifiable smart contract—implemented in Solidity—that ensures fraud resistance, fair cost allocation, and mutual compensation. Our approach establishes the first group-purchasing paradigm characterized by *no trusted third party*, *zero data leakage*, and *verifiable decision-making*. Empirical deployment demonstrates that the mechanism simultaneously preserves user privacy and significantly improves collective economic benefits and decision-making efficiency.

7 citationsRead paper
Recent publications

Latest Papers