Kitana: Efficient Data Augmentation Search for AutoML

by   Zezhou Huang, et al.

AutoML services provide a way for non-expert users to benefit from high-quality ML models without worrying about model design and deployment, in exchange for a charge per hour (21.252 for VertexAI). However, existing AutoML services are model-centric, in that they are limited to extracting features and searching for models from initial training data-they are only as effective as the initial training data quality. With the increasing volume of tabular data available, there is a huge opportunity for data augmentation. For instance, vertical augmentation adds predictive features, while horizontal augmentation adds examples. This augmented training data yields potentially much better AutoML models at a lower cost. However, existing systems either forgo the augmentation opportunities that provide poor models, or apply expensive augmentation searching techniques that drain users' budgets. Kitana is a data-centric AutoML system that also searches for new tabular datasets that can augment the tabular training data with new features and/or examples. Kitana manages a corpus of datasets, exposes an AutoML interface to users and searches for augmentation with datasets in the corpus to improve AutoML performance. To accelerate search, Kitana applies aggressive pre-computation to train a factorized proxy model and evaluate each candidate augmentation within 0.1s. Kitana also uses a cost model to limit the time spent on augmentation search, supports expressive data access controls, and performs request caching to benefit from past similar requests. Using a corpus of 518 open-source datasets, Kitana produces higher quality models than existing AutoML systems in orders of magnitude less time. Across different user requests, Kitana increases the model R2 from 0.16 to 0.66 while reducing the cost by >100x compared to the naive factorized learning and SOTA data augmentation search.


Greedy AutoAugment

A major problem in data augmentation is the number of possibilities in t...

Auctus: A Dataset Search Engine for Data Augmentation

Machine Learning models are increasingly being adopted in many applicati...

AutoDC: Automated data-centric processing

AutoML (automated machine learning) has been extensively developed in th...

Complex Wavelet SSIM based Image Data Augmentation

One of the biggest problems in neural learning networks is the lack of t...

ARTA: Collection and Classification of Ambiguous Requests and Thoughtful Actions

Human-assisting systems such as dialogue systems must take thoughtful, a...

On the Accuracy of CRNNs for Line-Based OCR: A Multi-Parameter Evaluation

We investigate how to train a high quality optical character recognition...

A New Tool for Efficiently Generating Quality Estimation Datasets

Building of data for quality estimation (QE) training is expensive and r...

Please sign up or login with your details

Forgot password? Click here to reset