research
∙
09/16/2023
Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference Using Sorted Fine-Tuning (SoFT)
The rapid advancement of large language models (LLMs) has revolutionized...
share
research
∙
09/01/2023
SortedNet, a Place for Every Network and Every Network in its Place: Towards a Generalized Solution for Training Many-in-One Neural Networks
As the size of deep learning models continues to grow, finding optimal m...
research
∙
04/22/2023
Recurrent Neural Networks and Long Short-Term Memory Networks: Tutorial and Survey
This is a tutorial paper on Recurrent Neural Network (RNN), Long Short-T...
research
∙
01/27/2023
Improved knowledge distillation by utilizing backward pass knowledge in neural networks
Knowledge distillation (KD) is one of the prominent techniques for model...
research
∙
12/12/2022
Continuation KD: Improved Knowledge Distillation through the Lens of Continuation Optimization
Knowledge Distillation (KD) has been extensively used for natural langua...
research
∙
12/12/2022
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging
Knowledge Distillation (KD) is a commonly used technique for improving t...
research
∙
10/14/2022
DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation
With the ever-growing size of pre-trained models (PMs), fine-tuning them...
research
∙
05/25/2022
Towards Understanding Label Regularization for Fine-tuning Pre-trained Language Models
Knowledge Distillation (KD) is a prominent neural model compression tech...
research
∙
05/14/2022
The Sky Above The Clouds
Technology ecosystems often undergo significant transformations as they ...
research
∙
03/17/2022
When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation
Data Augmentation (DA) is known to improve the generalizability of deep ...
research
∙
10/16/2021
Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher
With ever growing scale of neural models, knowledge distillation (KD) at...
research
∙
09/21/2021
Knowledge Distillation with Noisy Labels for Natural Language Understanding
Knowledge Distillation (KD) is extensively used to compress and deploy l...
research
∙
09/13/2021
KroneckerBERT: Learning Kronecker Decomposition for Pre-trained Language Models via Knowledge Distillation
The development of over-parameterized pre-trained language models has ma...
research
∙
09/13/2021
How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding
Knowledge Distillation (KD) is a model compression algorithm that helps ...
research
∙
06/27/2021
Legendre Deep Neural Network (LDNN) and its application for approximation of nonlinear Volterra Fredholm Hammerstein integral equations
Various phenomena in biology, physics, and engineering are modeled by di...
research
∙
06/27/2021
SymbolicGPT: A Generative Transformer Model for Symbolic Regression
Symbolic regression is the task of identifying a mathematical expression...
research
∙
05/28/2021
Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax
In Natural Language Processing (NLP), finding data augmentation techniqu...
research
∙
04/14/2021
Annealing Knowledge Distillation
Significant memory and computational requirements of large deep neural n...
research
∙
11/12/2020
Symbolically Solving Partial Differential Equations using Deep Learning
We describe a neural-based method for generating exact or approximate so...
research
∙
11/04/2020
A Neuro-Symbolic Method for Solving Differential and Functional Equations
When neural networks are used to solve differential equations, they usua...
research
∙
09/22/2020
Stochastic Neighbor Embedding with Gaussian and Student-t Distributions: Tutorial and Survey
Stochastic Neighbor Embedding (SNE) is a manifold learning and dimension...
research
∙
09/17/2020
Multidimensional Scaling, Sammon Mapping, and Isomap: Tutorial and Survey
Multidimensional Scaling (MDS) is one of the first fundamental manifold ...
research
∙
06/30/2020
Segmentation Approach for Coreference Resolution Task
In coreference resolution, it is important to consider all members of a ...
research
∙
04/17/2019
DeepNovoV2: Better de novo peptide sequencing with deep learning
We introduce DeepNovoV2, the state-of-the-art neural networks based mode...
research
∙
12/18/2018
Deep Variational Sufficient Dimensionality Reduction
We consider the problem of sufficient dimensionality reduction (SDR), wh...
research
∙
11/07/2018
SRP: Efficient class-aware embedding learning for large-scale data via supervised random projections
Supervised dimensionality reduction strategies have been of great intere...
research
∙
07/23/2018
Text Classification based on Multiple Block Convolutional Highways
In the Text Classification areas of Sentiment Analysis, Subjectivity/Obj...
research
∙
12/15/2017
A Berkeley View of Systems Challenges for AI
With the increasing commoditization of computer vision, speech recogniti...
research
∙
11/24/2017
JADE: Joint Autoencoders for Dis-Entanglement
The problem of feature disentanglement has been explored in the literatu...
research
∙
11/23/2017
Improving the Accuracy of Pre-trained Word Embeddings for Sentiment Analysis
Sentiment analysis is one of the well-known tasks and fast growing resea...
research
∙
08/25/2017
Deep Structure for end-to-end inverse rendering
Inverse rendering in a 3D format denoted to recovering the 3D properties...
research
∙
04/07/2017
Fast Spectral Clustering Using Autoencoders and Landmarks
In this paper, we introduce an algorithm for performing spectral cluster...
research
∙
02/10/2017
Generative Mixture of Networks
A generative model based on training deep architectures is proposed. The...
research
∙
04/25/2016
Semi-supervised Dictionary Learning Based on Hilbert-Schmidt Independence Criterion
In this paper, a novel semi-supervised dictionary learning and sparse re...
research
∙
03/06/2015
On the Invariance of Dictionary Learning and Sparse Representation to Projecting Data to a Discriminative Space
In this paper, it is proved that dictionary learning and sparse represen...
research
∙
02/20/2015
Supervised Dictionary Learning and Sparse Representation-A Review
Dictionary learning and sparse representation (DLSR) is a recent and suc...
research
∙
12/24/2013
A Fast Greedy Algorithm for Generalized Column Subset Selection
This paper defines a generalized column subset selection problem which i...
research
∙
07/12/2012
Supervised Texture Classification Using a Novel Compression-Based Similarity Measure
Supervised pixel-based texture classification is usually performed in th...
research
∙
07/10/2012