Detecting Sarcasm in Multimodal Social Platforms

by   Rossano Schifanella, et al.

Sarcasm is a peculiar form of sentiment expression, where the surface sentiment differs from the implied sentiment. The detection of sarcasm in social media platforms has been applied in the past mainly to textual utterances where lexical indicators (such as interjections and intensifiers), linguistic markers, and contextual information (such as user profiles, or past conversations) were used to detect the sarcastic tone. However, modern social media platforms allow to create multimodal messages where audiovisual content is integrated with the text, making the analysis of a mode in isolation partial. In our work, we first study the relationship between the textual and visual aspects in multimodal posts from three major social media platforms, i.e., Instagram, Tumblr and Twitter, and we run a crowdsourcing task to quantify the extent to which images are perceived as necessary by human annotators. Moreover, we propose two different computational frameworks to detect sarcasm that integrate the textual and visual modalities. The first approach exploits visual semantics trained on an external dataset, and concatenates the semantics features with state-of-the-art textual features. The second method adapts a visual neural network initialized with parameters trained on ImageNet to multimodal sarcastic posts. Results show the positive effect of combining modalities for the detection of sarcasm across platforms and methods.


page 1

page 5


A Multimodal Approach to Predict Social Media Popularity

Multiple modalities represent different aspects by which information is ...

Detecting Medical Misinformation on Social Media Using Multimodal Deep Learning

In 2019, outbreaks of vaccine-preventable diseases reached the highest n...

Exploring Emoji Usage and Prediction Through a Temporal Variation Lens

The frequent use of Emojis on social media platforms has created a new f...

Multimodal Emoji Prediction

Emojis are small images that are commonly included in social media text ...

"Subverting the Jewtocracy": Online Antisemitism Detection Using Multimodal Deep Learning

The exponential rise of online social media has enabled the creation, di...

VICSOM: VIsual Clues from SOcial Media for psychological assessment

Sharing multimodal information (typically images, videos or text) in Soc...

Feels Bad Man: Dissecting Automated Hateful Meme Detection Through the Lens of Facebook's Challenge

Internet memes have become a dominant method of communication; at the sa...

Please sign up or login with your details

Forgot password? Click here to reset