In Search of netUnicorn: A Data-Collection Platform to Develop Generalizable ML Models for Network Security Problems

by   Roman Beltiukov, et al.

The remarkable success of the use of machine learning-based solutions for network security problems has been impeded by the developed ML models' inability to maintain efficacy when used in different network environments exhibiting different network behaviors. This issue is commonly referred to as the generalizability problem of ML models. The community has recognized the critical role that training datasets play in this context and has developed various techniques to improve dataset curation to overcome this problem. Unfortunately, these methods are generally ill-suited or even counterproductive in the network security domain, where they often result in unrealistic or poor-quality datasets. To address this issue, we propose an augmented ML pipeline that leverages explainable ML tools to guide the network data collection in an iterative fashion. To ensure the data's realism and quality, we require that the new datasets should be endogenously collected in this iterative process, thus advocating for a gradual removal of data-related problems to improve model generalizability. To realize this capability, we develop a data-collection platform, netUnicorn, that takes inspiration from the classic "hourglass" model and is implemented as its "thin waist" to simplify data collection for different learning problems from diverse network environments. The proposed system decouples data-collection intents from the deployment mechanisms and disaggregates these high-level intents into smaller reusable, self-contained tasks. We demonstrate how netUnicorn simplifies collecting data for different learning problems from multiple network environments and how the proposed iterative data collection improves a model's generalizability.


page 1

page 2

page 3

page 4


Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning

A growing body of work shows that many problems in fairness, accountabil...

CARONTE: Crawling Adversarial Resources Over Non-Trusted, High-Profile Environments

The monitoring of underground criminal activities is often automated to ...

Designing Data: Proactive Data Collection and Iteration for Machine Learning

Lack of diversity in data collection has caused significant failures in ...

An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context

As use of data driven technologies spreads, software engineers are more ...

Satyam: Democratizing Groundtruth for Machine Vision

The democratization of machine learning (ML) has led to ML-based machine...

FPIC: A Novel Semantic Dataset for Optical PCB Assurance

The continued outsourcing of printed circuit board (PCB) fabrication to ...

Scalable and Accurate Test Case Prioritization in Continuous Integration Contexts

Continuous Integration (CI) requires efficient regression testing to ens...

Please sign up or login with your details

Forgot password? Click here to reset