Exploring the Limits of Transfer Learning with Unified Model in the Cybersecurity Domain

by   Kuntal Kumar Pal, et al.

With the increase in cybersecurity vulnerabilities of software systems, the ways to exploit them are also increasing. Besides these, malware threats, irregular network interactions, and discussions about exploits in public forums are also on the rise. To identify these threats faster, to detect potentially relevant entities from any texts, and to be aware of software vulnerabilities, automated approaches are necessary. Application of natural language processing (NLP) techniques in the Cybersecurity domain can help in achieving this. However, there are challenges such as the diverse nature of texts involved in the cybersecurity domain, the unavailability of large-scale publicly available datasets, and the significant cost of hiring subject matter experts for annotations. One of the solutions is building multi-task models that can be trained jointly with limited data. In this work, we introduce a generative multi-task model, Unified Text-to-Text Cybersecurity (UTS), trained on malware reports, phishing site URLs, programming code constructs, social media data, blogs, news articles, and public forum posts. We show UTS improves the performance of some cybersecurity datasets. We also show that with a few examples, UTS can be adapted to novel unseen tasks and the nature of data


Security Vulnerability Detection Using Deep Learning Natural Language Processing

Detecting security vulnerabilities in software before they are exploited...

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Transfer learning, where a model is first pre-trained on a data-rich tas...

Improving the Generalizability of Text-Based Emotion Detection by Leveraging Transformers with Psycholinguistic Features

In recent years, there has been increased interest in building predictiv...

An Emotion-Aware Multi-Task Approach to Fake News and Rumour Detection using Transfer Learning

Social networking sites, blogs, and online articles are instant sources ...

Extreme Multi-Domain, Multi-Task Learning With Unified Text-to-Text Transfer Transformers

Text-to-text transformers have shown remarkable success in the task of m...

Generating Informative Conclusions for Argumentative Texts

The purpose of an argumentative text is to support a certain conclusion....

Exploring a Unified Sequence-To-Sequence Transformer for Medical Product Safety Monitoring in Social Media

Adverse Events (AE) are harmful events resulting from the use of medical...

Please sign up or login with your details

Forgot password? Click here to reset