Institution profile

Ningxia University

Academic institutionasia · cn
Official website
Research library7linked papers
Opportunities0open roles
Selected work

Representative Papers

PJ-RoPE: A Fourier-Jet-Affine Position Space for Relative Attention

Jun 03, 2026

This work proposes PJ-RoPE, a novel learnable relative positional encoding framework that unifies Fourier phase, finite-order jets (including those with repeated roots), and unit-root affine recency within the algebraic structure of constant-coefficient difference modules. By decoupling feature transformation from bias kernels, introducing LC/rapidity coordinates, and incorporating an adaptive sector-diagnostic mechanism, the method stabilizes high-order jets and reveals task-specific preferences for positional structures. Experiments demonstrate that distinct tasks exhibit strong affinities for particular positional sectors; in small-scale language modeling, a clear boundary emerges between affine and recency-based encodings; in music sequence modeling, LC/affine variants achieve superior performance and implicitly encode higher-order corrections; and while LC coordinates enhance scale stability, they entail a trade-off in phase resolution.

0 citationsRead paper

Jordan-RoPE: Non-Semisimple Relative Positional Encoding via Complex Jordan Blocks

May 05, 2026

Relative positional encodings determine which functions of query-key lag can enter the primitive attention logit. RoPE supplies a rotary phase, while ALiBi supplies an additive distance bias. Motivated by group-theoretic views of linear translation-invariant positional encodings, we study a non-semisimple case in which a complex rotary eigenvalue and a nilpotent response live in the same defective Jordan block. The resulting relative operator generates oscillatory-polynomial features such as $e^{-γd}\cos(ωd)$, $e^{-γd}\sin(ωd)$, $d e^{-γd}\cos(ωd)$, and $d e^{-γd}\sin(ωd)$, for causal lag $d=i-j\geq 0$. Thus the construction realizes a distance-modulated phase basis $d e^{iωd}$, rather than merely adding a separate distance channel to RoPE. We formulate Exact Jordan-RoPE as a non-semisimple one-parameter representation, give its real block form, and specify the contragredient query action required by non-orthogonal positional maps. We also distinguish this exact representation from stabilized variants whose bounded shear improves numerical behavior but breaks the exact group law. Kernel-level diagnostics and a Jordan-friendly synthetic language-model task show that the coupled Jordan basis is useful when the target contains distance-modulated phase interactions. On a small WikiText-103 byte language model, a scaled-exact variant improves over RoPE and direct-sum baselines within the Jordan family, while RoPE+ALiBi remains strongest overall. The evidence is structural rather than a broad performance claim.

0 citationsRead paper

Latent-Info and Low-Dimensional Learning for Human Mesh Recovery and Parallel Optimization

Oct 20, 2025

Existing 3D human mesh reconstruction methods suffer from limb misalignment, insufficient local geometric detail, and high computational overhead—especially in complex scenes. To address these issues, we propose a two-stage lightweight and efficient network. First, we decompose image features into high- and low-frequency components and construct a hybrid implicit frequency-domain representation to jointly model global structure and local details. Second, we introduce a low-dimensional mesh-pose interaction mechanism—replacing costly vertex-level attention—to enable coupled optimization of pose and shape. Leveraging implicit information mining, multi-scale feature aggregation, and parallel optimization, our approach significantly reduces computational complexity while improving reconstruction accuracy. Experiments demonstrate state-of-the-art performance across multiple large-scale benchmarks, with substantial improvements in limb alignment and fine-grained geometric fidelity, and a 42% speedup in inference time.

0 citationsRead paper
Recent publications

Latest Papers

PJ-RoPE: A Fourier-Jet-Affine Position Space for Relative Attention

Jun 03, 2026

This work proposes PJ-RoPE, a novel learnable relative positional encoding framework that unifies Fourier phase, finite-order jets (including those with repeated roots), and unit-root affine recency within the algebraic structure of constant-coefficient difference modules. By decoupling feature transformation from bias kernels, introducing LC/rapidity coordinates, and incorporating an adaptive sector-diagnostic mechanism, the method stabilizes high-order jets and reveals task-specific preferences for positional structures. Experiments demonstrate that distinct tasks exhibit strong affinities for particular positional sectors; in small-scale language modeling, a clear boundary emerges between affine and recency-based encodings; in music sequence modeling, LC/affine variants achieve superior performance and implicitly encode higher-order corrections; and while LC coordinates enhance scale stability, they entail a trade-off in phase resolution.

0 citationsRead paper

Jordan-RoPE: Non-Semisimple Relative Positional Encoding via Complex Jordan Blocks

May 05, 2026

Relative positional encodings determine which functions of query-key lag can enter the primitive attention logit. RoPE supplies a rotary phase, while ALiBi supplies an additive distance bias. Motivated by group-theoretic views of linear translation-invariant positional encodings, we study a non-semisimple case in which a complex rotary eigenvalue and a nilpotent response live in the same defective Jordan block. The resulting relative operator generates oscillatory-polynomial features such as $e^{-γd}\cos(ωd)$, $e^{-γd}\sin(ωd)$, $d e^{-γd}\cos(ωd)$, and $d e^{-γd}\sin(ωd)$, for causal lag $d=i-j\geq 0$. Thus the construction realizes a distance-modulated phase basis $d e^{iωd}$, rather than merely adding a separate distance channel to RoPE. We formulate Exact Jordan-RoPE as a non-semisimple one-parameter representation, give its real block form, and specify the contragredient query action required by non-orthogonal positional maps. We also distinguish this exact representation from stabilized variants whose bounded shear improves numerical behavior but breaks the exact group law. Kernel-level diagnostics and a Jordan-friendly synthetic language-model task show that the coupled Jordan basis is useful when the target contains distance-modulated phase interactions. On a small WikiText-103 byte language model, a scaled-exact variant improves over RoPE and direct-sum baselines within the Jordan family, while RoPE+ALiBi remains strongest overall. The evidence is structural rather than a broad performance claim.

0 citationsRead paper

Latent-Info and Low-Dimensional Learning for Human Mesh Recovery and Parallel Optimization

Oct 20, 2025

Existing 3D human mesh reconstruction methods suffer from limb misalignment, insufficient local geometric detail, and high computational overhead—especially in complex scenes. To address these issues, we propose a two-stage lightweight and efficient network. First, we decompose image features into high- and low-frequency components and construct a hybrid implicit frequency-domain representation to jointly model global structure and local details. Second, we introduce a low-dimensional mesh-pose interaction mechanism—replacing costly vertex-level attention—to enable coupled optimization of pose and shape. Leveraging implicit information mining, multi-scale feature aggregation, and parallel optimization, our approach significantly reduces computational complexity while improving reconstruction accuracy. Experiments demonstrate state-of-the-art performance across multiple large-scale benchmarks, with substantial improvements in limb alignment and fine-grained geometric fidelity, and a 42% speedup in inference time.

0 citationsRead paper