An empirical evaluation of attention-based multi-head models for improved turbofan engine remaining useful life prediction

by   Abiodun Ayodeji, et al.

A single unit (head) is the conventional input feature extractor in deep learning architectures trained on multivariate time series signals. The importance of the fixed-dimensional vector representation generated by the single-head network has been demonstrated for industrial machinery condition monitoring and predictive maintenance. However, processing heterogeneous sensor signals with a single head may result in a model that cannot explicitly account for the diversity in time-varying multivariate inputs. This work extends the conventional single-head deep learning models to a more robust form by developing context-specific heads to independently capture the inherent pattern of each sensor reading in multivariate time series signals. Using the turbofan aircraft engine benchmark dataset (CMAPSS), an extensive experiment is performed to verify the effectiveness and benefits of multi-head fully connected neurons, recurrent networks, convolution network, the transformer-style stand-alone attention network, and their variants for remaining useful life estimation. Moreover, the effect of different attention mechanisms on the multi-head models is also evaluated. In addition, each architecture's relative advantage and computational overhead are analyzed. Results show that utilizing the attention layer is task-sensitive and model-dependent, as it does not provide consistent improvement across the models investigated. The result is further compared with five state-of-the-art models, and the comparison shows that a relatively simple multi-head architecture performs better than the state-of-the-art models. The results presented in this study demonstrate the importance of multi-head models and attention mechanisms to improved understanding of the remaining useful life of industrial assets.


page 14

page 15

page 16

page 17

page 18

page 19


Multi-Dimensional Self Attention based Approach for Remaining Useful Life Estimation

Remaining Useful Life (RUL) estimation plays a critical role in Prognost...

Remaining Useful Life Estimation Under Uncertainty with Causal GraphNets

In this work, a novel approach for the construction and training of time...

A Transformer-based Framework For Multi-variate Time Series: A Remaining Useful Life Prediction Use Case

In recent times, Large Language Models (LLMs) have captured a global spo...

Graph Neural Networks for Leveraging Industrial Equipment Structure: An application to Remaining Useful Life Estimation

Automated equipment health monitoring from streaming multisensor time-se...

Genetically Optimized Prediction of Remaining Useful Life

The application of remaining useful life (RUL) prediction has taken grea...

Dual Aspect Self-Attention based on Transformer for Remaining Useful Life Prediction

Remaining useful life prediction (RUL) is one of the key technologies of...

Latent Temporal Flows for Multivariate Analysis of Wearables Data

Increased use of sensor signals from wearable devices as rich sources of...

Please sign up or login with your details

Forgot password? Click here to reset