How to Evaluate Semantic Communications for Images with ViTScore Metric?

by   Tingting Zhu, et al.

Semantic communications (SC) have been expected to be a new paradigm shifting to catalyze the next generation communication, whose main concerns shift from accurate bit transmission to effective semantic information exchange in communications. However, the previous and widely-used metrics for images are not applicable to evaluate the image semantic similarity in SC. Classical metrics to measure the similarity between two images usually rely on the pixel level or the structural level, such as the PSNR and the MS-SSIM. Straightforwardly using some tailored metrics based on deep-learning methods in CV community, such as the LPIPS, is infeasible for SC. To tackle this, inspired by BERTScore in NLP community, we propose a novel metric for evaluating image semantic similarity, named Vision Transformer Score (ViTScore). We prove theoretically that ViTScore has 3 important properties, including symmetry, boundedness, and normalization, which make ViTScore convenient and intuitive for image measurement. To evaluate the performance of ViTScore, we compare ViTScore with 3 typical metrics (PSNR, MS-SSIM, and LPIPS) through 5 classes of experiments. Experimental results demonstrate that ViTScore can better evaluate the image semantic similarity than the other 3 typical metrics, which indicates that ViTScore is an effective performance metric when deployed in SC scenarios.


page 1

page 2

page 5

page 7

page 8

page 11


Semantic Communications With AI Tasks

A radical paradigm shift of wireless networks from “connected things” to...

Seq2Seq-SC: End-to-End Semantic Communication Systems with Pre-trained Language Model

While semantic communication is expected to bring unprecedented communic...

Towards Semantic Communications: Deep Learning-Based Image Semantic Coding

Semantic communications has received growing interest since it can remar...

Large AI Model-Based Semantic Communications

Semantic communication (SC) is an emerging intelligent paradigm, offerin...

Transceiver Cooperative Learning-aided Semantic Communications Against Mismatched Background Knowledge Bases

Semantic communications learned on background knowledge bases (KBs) have...

Learning Shadow Correspondence for Video Shadow Detection

Video shadow detection aims to generate consistent shadow predictions am...

Image Restoration under Semantic Communications

Semantic communication has emerged as the breakthrough beyond the Shanno...

Please sign up or login with your details

Forgot password? Click here to reset