End-to-End Multi-Task Deep Learning and Model Based Control Algorithm for Autonomous Driving

by   Der-Hau Lee, et al.

End-to-end driving with a deep learning neural network (DNN) has become a rapidly growing paradigm of autonomous driving in industry and academia. Yet safety measures and interpretability still pose challenges to this paradigm. We propose an end-to-end driving algorithm that integrates multi-task DNN, path prediction, and control models in a pipeline of data flow from sensory devices through these models to driving decisions. It provides quantitative measures to evaluate the holistic, dynamic, and real-time performance of end-to-end driving systems, and thus allows to quantify their safety and interpretability. The DNN is a modified UNet, a well known encoder-decoder neural network of semantic segmentation. It consists of one segmentation, one regression, and two classification tasks for lane segmentation, path prediction, and vehicle controls. We present three variants of the modified UNet architecture having different complexities, compare them on different tasks in four static measures for both single and multi-task (MT) architectures, and then identify the best one by two additional dynamic measures in real-time simulation. We also propose a learning- and model-based longitudinal controller using model predictive control method. With the Stanley lateral controller, our results show that MTUNet outperforms an earlier modified UNet in terms of curvature and lateral offset estimation on curvy roads at normal speed, which has been tested in a real car driving on real roads.


page 3

page 4

page 6


End-to-End Deep Learning of Lane Detection and Path Prediction for Real-Time Autonomous Driving

We propose an end-to-end three-task convolutional neural network (3TCNN)...

Hierarchical Multi-task Deep Neural Network Architecture for End-to-End Driving

A novel hierarchical Deep Neural Network (DNN) model is presented to add...

Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Control

Deep neural network controllers for autonomous driving have recently ben...

Learning a CNN-based End-to-End Controller for a Formula SAE Racecar

We present a set of CNN-based end-to-end models for controls of a Formul...

Closing the gap towards end-to-end autonomous vehicle system

Designing a driving policy for autonomous vehicles is a difficult task. ...

NAIS: Neural Architecture and Implementation Search and its Applications in Autonomous Driving

The rapidly growing demands for powerful AI algorithms in many applicati...

Application of End-to-End Deep Learning in Wireless Communications Systems

Deep learning is a potential paradigm changer for the design of wireless...

Please sign up or login with your details

Forgot password? Click here to reset