Parseval Proximal Neural Networks
share
The aim of this paper is twofold. First, we show that a certain concatenation of a proximity operator with an affine operator is again a proximity operator on a suitable Hilbert space. Second, we use our findings to establish so-called proximal neural networks (PNNs) and stable Parseval (frame) proximal neural networks (PPNNs). Let H and K be real Hilbert spaces, b ∈K and T ∈B (H,K) a linear operator with closed range and Moore-Penrose inverse T^†. Based on the well-known characterization of proximity operators by Moreau, we prove that for any proximity operator ProxK→K the operator T^† Prox ( T · + b) is a proximity operator on H equipped with a suitable norm. In particular, it follows for the frequently applied soft shrinkage operator Prox = S_λℓ_2 →ℓ_2 and any frame analysis operator TH→ℓ_2, that the frame shrinkage operator T^† S_λ T is a proximity operator in a suitable Hilbert space. Further, the concatenation of proximity operators on R^d equipped with different norms establishes a PNN. If the network arises from Parseval frame analysis or synthesis operators, it forms an averaged operator, called PPNN. The involved linear operators, respectively their transposed operators, are in a Stiefel manifold, so that minimization methods on Stiefel manifolds can be applied for training such networks. Finally, some proof-of-the concept examples demonstrate the performance of PPNNs.
READ FULL TEXT