GT-PCQA: Geometry-Texture Decoupled Point Cloud Quality Assessment with MLLM

2026-03-16

AI Summary

Existing image quality assessment methods based on multimodal large language models (MLLMs) struggle to generalize directly to point cloud quality assessment, primarily due to the scarcity of point cloud data and the insensitivity of MLLMs to geometric distortions. To address this, this work proposes GT-PCQA, a no-reference point cloud quality assessment framework that introduces a novel geometry-texture decoupling mechanism and a dual-prompt strategy, combined with alternating optimization to mitigate the MLLM’s inherent texture bias. Furthermore, it devises a 2D–3D joint relative quality comparison training paradigm, leveraging cross-modal instruction tuning to efficiently utilize limited annotated data. Experimental results demonstrate that the proposed method achieves state-of-the-art performance across multiple point cloud quality assessment benchmarks and exhibits strong generalization capabilities.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

With the rapid advancement of Multi-modal Large Language Models (MLLMs), MLLM-based Image Quality Assessment (IQA) methods have shown promising generalization. However, directly extending these MLLM-based IQA methods to PCQA remains challenging. On the one hand, existing PCQA datasets are limited in scale, which hinders stable and effective instruction tuning of MLLMs. On the other hand, due to large-scale image-text pretraining, MLLMs tend to rely on texture-dominant reasoning and are insufficiently sensitive to geometric structural degradations that are critical for PCQA. To address these gaps, we propose a novel MLLM-based no-reference PCQA framework, termed GT-PCQA, which is built upon two key strategies. First, to enable stable and effective instruction tuning under scarce PCQA supervision, a 2D-3D joint training strategy is proposed. This strategy formulates PCQA as a relative quality comparison problem to unify large-scale IQA datasets with limited PCQA datasets. It incorporates a parameter-efficient Low-Rank Adaptation (LoRA) scheme to support instruction tuning. Second, a geometry-texture decoupling strategy is presented, which integrates a dual-prompt mechanism with an alternating optimization scheme to mitigate the inherent texture-dominant bias of pre-trained MLLMs, while enhancing sensitivity to geometric structural degradations. Extensive experiments demonstrate that GT-PCQA achieves competitive performance and exhibits strong generalization.

Problem

• Point Cloud Quality Assessment
• Multi-modal Large Language Models
• Geometry-Texture Decoupling
• Instruction Tuning
• Geometric Degradation

Innovation

• Geometry-Texture Decoupling
• 2D-3D Joint Training
• MLLM-based PCQA
• LoRA
• Dual-Prompt Mechanism