Learning with Known Operators reduces Maximum Training Error Bounds

by   Andreas K. Maier, et al.

We describe an approach for incorporating prior knowledge into machine learning algorithms. We aim at applications in physics and signal processing in which we know that certain operations must be embedded into the algorithm. Any operation that allows computation of a gradient or sub-gradient towards its inputs is suited for our framework. We derive a maximal error bound for deep nets that demonstrates that inclusion of prior knowledge results in its reduction. Furthermore, we also show experimentally that known operators reduce the number of free parameters. We apply this approach to various tasks ranging from CT image reconstruction over vessel segmentation to the derivation of previously unknown imaging algorithms. As such the concept is widely applicable for many researchers in physics, imaging, and signal processing. We assume that our analysis will support further investigation of known operators in other fields of physics, imaging, and signal processing.


Precision Learning: Towards Use of Known Operators in Neural Networks

In this paper, we consider the use of prior knowledge within neural netw...

Theory and Algorithms for Pulse Signal Processing

The integrate and fire converter transforms an analog signal into train ...

Fuzzy-based Propagation of Prior Knowledge to Improve Large-Scale Image Analysis Pipelines

Many automatically analyzable scientific questions are well-posed and of...

Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning

Stochastic approximation (SA) is a classical algorithm that has had sinc...

Kohn-Sham equations as regularizer: building prior knowledge into machine-learned physics

Including prior knowledge is important for effective machine learning mo...

Deep Neural Networks to Recover Unknown Physical Parameters from Oscillating Time Series

Deep neural networks (DNNs) are widely used in pattern-recognition tasks...

Tao General Differential and Difference: Theory and Application

Modern numerical analysis is executed on discrete data, of which numeric...

Please sign up or login with your details

Forgot password? Click here to reset