Multi-Layer Perceptron
Physics-Informed Neural Networks (PINNs)
DeepONet: Operator Learning with Neural Networks
Differentiable Simulation
Graph Neural Network
Lab Setup

The Universal Approximation Theorem for Operators

The Breakthrough Theorem

Just as the Universal Approximation Theorem tells us neural networks can approximate functions, there's a remarkable extension:

Theorem (Chen & Chen, 1995): Neural networks can approximate operators that map functions to functions!

Mathematical Statement: For any continuous operator \( \mathcal{G}: V \subset C(K_1) \rightarrow C(K_2) \) and \( \epsilon > 0 \), there exist constants such that:

\[ \left|\mathcal{G}(u)(y) - \sum_{k=1}^p \underbrace{\sum_{i=1}^n c_i^k \sigma\left(\sum_{j=1}^m \xi_{ij}^k u(x_j) + \theta_i^k\right)}_{\text{Branch Network}} \underbrace{\sigma(w_k \cdot y + \zeta_k)}_{\text{Trunk Network}}\right| < \epsilon \]

Decoding the Theorem

This looks complex, but the insight is beautiful:

  1. Branch Network: Processes function \( u \) sampled at sensor points \( \{x_j\} \)
  2. Trunk Network: Processes output coordinates \( y \)
  3. Combination: Multiply branch and trunk outputs, then sum
Key insight:

Any operator can be written as:

\[ \mathcal{G}(u)(y) \approx \sum_{k=1}^p b_k(u) \cdot t_k(y) \]

where \( b_k \) depends only on the input function and \( t_k \) depends only on the output location!

Back
 
Next
©  |   Cornell University    |   Center for Advanced Computing    |   Copyright Statement    |   Access Statement
CVW material development is supported by NSF OAC awards 1854828, 2321040, 2323116 (UT Austin) and 2005506 (Indiana University)