Construction of Large-Scale Misinformation Labeled Datasets from Social Media Discourse using Label Refinement

by   Karishma Sharma, et al.

Malicious accounts spreading misinformation has led to widespread false and misleading narratives in recent times, especially during the COVID-19 pandemic, and social media platforms struggle to eliminate these contents rapidly. This is because adapting to new domains requires human intensive fact-checking that is slow and difficult to scale. To address this challenge, we propose to leverage news-source credibility labels as weak labels for social media posts and propose model-guided refinement of labels to construct large-scale, diverse misinformation labeled datasets in new domains. The weak labels can be inaccurate at the article or social media post level where the stance of the user does not align with the news source or article credibility. We propose a framework to use a detection model self-trained on the initial weak labels with uncertainty sampling based on entropy in predictions of the model to identify potentially inaccurate labels and correct for them using self-supervision or relabeling. The framework will incorporate social context of the post in terms of the community of its associated user for surfacing inaccurate labels towards building a large-scale dataset with minimum human effort. To provide labeled datasets with distinction of misleading narratives where information might be missing significant context or has inaccurate ancillary details, the proposed framework will use the few labeled samples as class prototypes to separate high confidence samples into false, unproven, mixture, mostly false, mostly true, true, and debunk information. The approach is demonstrated for providing a large-scale misinformation dataset on COVID-19 vaccines.


page 1

page 2

page 3

page 4


Detecting Fake News with Weak Social Supervision

Limited labeled data is becoming the largest bottleneck for supervised l...

AMUSED: An Annotation Framework of Multi-modal Social Media Data

In this paper, we present a semi-automated framework called AMUSED for g...

Learning Class-specific Word Representations for Early Detection of Hoaxes in Social Media

As people increasingly use social media as a source for news consumption...

SRLF: A Stance-aware Reinforcement Learning Framework for Content-based Rumor Detection on Social Media

The rapid development of social media changes the lifestyle of people an...

Using Social Media Images for Building Function Classification

Urban land use on a building instance level is crucial geo-information f...

A study on information disorders on social networks during the Chilean social outbreak and COVID-19 pandemic

Information disorders on social media can have a significant impact on c...

A Deep Metric Learning Approach to Account Linking

We consider the task of linking social media accounts that belong to the...

Please sign up or login with your details

Forgot password? Click here to reset