Institution profile

Canva

Industry researchaustralasia · au
Official website
Research library13linked papers
Opportunities0open roles
Selected work

Representative Papers

Learning in Curved Weight Space:Exponential-Linear Weight Reparameterization for Improved Optimization

Jul 10, 2026

This work addresses the inefficiency in traditional neural network optimization, where additive weight updates induce imbalanced relative perturbations across weights of differing magnitudes. To mitigate this, the authors propose a hybrid exponential-linear reparameterization of weights that integrates a sign-aware symmetric exponential pathway with an identity linear pathway. This construction, augmented with learnable scale, curvature, and offset parameters, induces a curved weight geometry wherein optimization step sizes scale proportionally with weight magnitudes. Coupled with a mismatched initialization strategy to encourage early symmetry breaking, the method achieves equivalent validation loss on OpenWebText using 1.32–1.49× fewer training steps across various Transformer architectures, with particularly pronounced gains for wide models.

0 citationsRead paper

DiffusionBench: On Holistic Evaluation of Diffusion Transformers

Jun 23, 2026

This work addresses the limitation of existing DiT research, which overly relies on ImageNet class-conditional generation as a sole evaluation setting and fails to reflect model performance in broader applications such as text-to-image (T2I) synthesis. To this end, we propose NanoGen, a unified training and evaluation framework that supports diverse diffusion approaches—including RAE, VAE, pixel-space, and MeanFlow—across both ImageNet and T2I tasks with only a 12-line configuration switch. We further introduce DiffusionBench, a comprehensive benchmark encompassing both task types. Experiments across 21 latent diffusion models reveal a significant negative correlation between ImageNet and T2I performance, with Pearson correlation coefficients ranging from −0.377 to −0.580, underscoring the misleading nature of single-task evaluation and validating the necessity and effectiveness of DiffusionBench.

0 citationsRead paper
Recent publications

Latest Papers

Learning in Curved Weight Space:Exponential-Linear Weight Reparameterization for Improved Optimization

Jul 10, 2026

This work addresses the inefficiency in traditional neural network optimization, where additive weight updates induce imbalanced relative perturbations across weights of differing magnitudes. To mitigate this, the authors propose a hybrid exponential-linear reparameterization of weights that integrates a sign-aware symmetric exponential pathway with an identity linear pathway. This construction, augmented with learnable scale, curvature, and offset parameters, induces a curved weight geometry wherein optimization step sizes scale proportionally with weight magnitudes. Coupled with a mismatched initialization strategy to encourage early symmetry breaking, the method achieves equivalent validation loss on OpenWebText using 1.32–1.49× fewer training steps across various Transformer architectures, with particularly pronounced gains for wide models.

0 citationsRead paper

DiffusionBench: On Holistic Evaluation of Diffusion Transformers

Jun 23, 2026

This work addresses the limitation of existing DiT research, which overly relies on ImageNet class-conditional generation as a sole evaluation setting and fails to reflect model performance in broader applications such as text-to-image (T2I) synthesis. To this end, we propose NanoGen, a unified training and evaluation framework that supports diverse diffusion approaches—including RAE, VAE, pixel-space, and MeanFlow—across both ImageNet and T2I tasks with only a 12-line configuration switch. We further introduce DiffusionBench, a comprehensive benchmark encompassing both task types. Experiments across 21 latent diffusion models reveal a significant negative correlation between ImageNet and T2I performance, with Pearson correlation coefficients ranging from −0.377 to −0.580, underscoring the misleading nature of single-task evaluation and validating the necessity and effectiveness of DiffusionBench.

0 citationsRead paper