Universal Approximation Theorem: The Theoretical Foundation
Universal Approximation Theorem (Cybenko, 1989):
A single hidden layer network with sufficient neurons can approximate any continuous function to arbitrary accuracy.
\[ F(x) = \sum_{i=1}^{N} w_i \sigma(v_i x + b_i) + w_0 \]
Mathematical statement: For any continuous \(f: [0,1] \to \mathbb{R}\) and \(\epsilon > 0\), there exists \(N\) and parameters such that \(|F(x) - f(x)| < \epsilon\) for all \(x \in [0,1]\).
Key questions:
- How many neurons \(N\) do we need?
- Is this practical?
- Can we verify this experimentally?
©
|
Cornell University
|
Center for Advanced Computing
|
Copyright Statement
|
Access Statement
CVW material development is supported by NSF OAC awards 1854828, 2321040, 2323116 (UT Austin) and 2005506 (Indiana University)
CVW material development is supported by NSF OAC awards 1854828, 2321040, 2323116 (UT Austin) and 2005506 (Indiana University)