Factored Temporal Sigmoid Belief Networks for Sequence Learning

by   Jiaming Song, et al.

Deep conditional generative models are developed to simultaneously learn the temporal dependencies of multiple sequences. The model is designed by introducing a three-way weight tensor to capture the multiplicative interactions between side information and sequences. The proposed model builds on the Temporal Sigmoid Belief Network (TSBN), a sequential stack of Sigmoid Belief Networks (SBNs). The transition matrices are further factored to reduce the number of parameters and improve generalization. When side information is not available, a general framework for semi-supervised learning based on the proposed model is constituted, allowing robust sequence classification. Experimental results show that the proposed approach achieves state-of-the-art predictive and classification performance on sequential data, and has the capacity to synthesize sequences, with controlled style transitioning and blending.


page 1

page 2

page 3

page 4


Deep Temporal Sigmoid Belief Networks for Sequence Modeling

Deep dynamic generative models are developed to learn sequential depende...

Improved Classification Based on Deep Belief Networks

For better classification generative models are used to initialize the m...

Dyna Planning using a Feature Based Generative Model

Dyna-style reinforcement learning is a powerful approach for problems wh...

A Cross-Sentence Latent Variable Model for Semi-Supervised Text Sequence Matching

We present a latent variable model for predicting the relationship betwe...

Exponentially Increasing the Capacity-to-Computation Ratio for Conditional Computation in Deep Learning

Many state-of-the-art results obtained with deep networks are achieved w...

Learning deep autoregressive models for hierarchical data

We propose a model for hierarchical structured data as an extension to t...

Tensorizing GAN with High-Order Pooling for Alzheimer's Disease Assessment

It is of great significance to apply deep learning for the early diagnos...

Please sign up or login with your details

Forgot password? Click here to reset