MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment

by   Zicheng Zhang, et al.

The visual quality of point clouds has been greatly emphasized since the ever-increasing 3D vision applications are expected to provide cost-effective and high-quality experiences for users. Looking back on the development of point cloud quality assessment (PCQA) methods, the visual quality is usually evaluated by utilizing single-modal information, i.e., either extracted from the 2D projections or 3D point cloud. The 2D projections contain rich texture and semantic information but are highly dependent on viewpoints, while the 3D point clouds are more sensitive to geometry distortions and invariant to viewpoints. Therefore, to leverage the advantages of both point cloud and projected image modalities, we propose a novel no-reference point cloud quality assessment (NR-PCQA) metric in a multi-modal fashion. In specific, we split the point clouds into sub-models to represent local geometry distortions such as point shift and down-sampling. Then we render the point clouds into 2D image projections for texture feature extraction. To achieve the goals, the sub-models and projected images are encoded with point-based and image-based neural networks. Finally, symmetric cross-modal attention is employed to fuse multi-modal quality-aware information. Experimental results show that our approach outperforms all compared state-of-the-art methods and is far ahead of previous NR-PCQA methods, which highlights the effectiveness of the proposed method.


page 1

page 3


A No-reference Quality Assessment Metric for Point Cloud Based on Captured Video Sequences

Point cloud is one of the most widely used digital formats of 3D models,...

Treating Point Cloud as Moving Camera Videos: A No-Reference Quality Assessment Metric

Point cloud is one of the most widely used digital representation format...

No-reference Point Cloud Geometry Quality Assessment Based on Pairwise Rank Learning

Objective geometry quality assessment of point clouds is essential to ev...

Multi-Modal Loop Closing in Unstructured Planetary Environments with Visually Enriched Submaps

Future planetary missions will rely on rovers that can autonomously expl...

A generalized Hausdorff distance based quality metric for point cloud geometry

Reliable quality assessment of decoded point cloud geometry is essential...

A Point-to-Distribution Joint Geometry and Color Metric for Point Cloud Quality Assessment

Point clouds (PCs) are a powerful 3D visual representation paradigm for ...

Point Cloud-based Proactive Link Quality Prediction for Millimeter-wave Communications

This study demonstrates the feasibility of point cloud-based proactive l...

Please sign up or login with your details

Forgot password? Click here to reset