An Empirical Study on Learning and Improving the Search Objective for Unsupervised Paraphrasing

by   Weikai Steven Lu, et al.

Research in unsupervised text generation has been gaining attention over the years. One recent approach is local search towards a heuristically defined objective, which specifies language fluency, semantic meanings, and other task-specific attributes. Search in the sentence space is realized by word-level edit operations including insertion, replacement, and deletion. However, such objective function is manually designed with multiple components. Although previous work has shown maximizing this objective yields good performance in terms of true measure of success (i.e. BLEU and iBLEU), the objective landscape is considered to be non-smooth with significant noises, posing challenges for optimization. In this dissertation, we address the research problem of smoothing the noise in the heuristic search objective by learning to model the search dynamics. Then, the learned model is combined with the original objective function to guide the search in a bootstrapping fashion. Experimental results show that the learned models combined with the original search objective can indeed provide a smoothing effect, improving the search performance by a small margin.


page 1

page 2

page 3

page 4


Search and Learning for Unsupervised Text Generation

With the advances of deep learning techniques, text generation is attrac...

Homotopic Convex Transformation: A New Method to Smooth the Landscape of the Traveling Salesman Problem

This paper proposes a novel landscape smoothing method for the symmetric...

Unsupervised Text Generation by Learning from Search

In this work, we present TGLS, a novel framework to unsupervised Text Ge...

Error Analysis of Surrogate Models Constructed through Operations on Sub-models

Model-based methods are popular in derivative-free optimization (DFO). I...

Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning

Sentence compression reduces the length of text by removing non-essentia...

An Incremental Evaluation Mechanism for the Critical Node Problem

The Critical Node Problem (CNP) is to identify a subset of nodes in a gr...

Information Utilization Ratio in Heuristic Optimization Algorithms

Heuristic algorithms are able to optimize objective functions efficientl...

Please sign up or login with your details

Forgot password? Click here to reset