Functional Analysis & Hilbert Spaces

Functional analysis is the study of vector spaces endowed with a limit-related structure (such as a norm or inner product) and the linear operators acting upon them. It generalizes the concepts of linear algebra to infinite-dimensional spaces, providing a rigorous framework for differential equations, quantum mechanics, and numerical analysis.

1. From Finite to Infinite Dimensions

In finite-dimensional linear algebra, we are accustomed to spaces like ${\mathbb{R}}^n$, where every linear operator can be represented as a matrix, and all norms are equivalent. However, most spaces of functions (like the space of continuous functions on an interval) are infinite-dimensional.

A key distinction is the concept of a basis:

• Hamel Basis: A set of vectors such that every $x\in X$ is a finite linear combination of basis vectors.
• Schauder Basis: Allows for infinite linear combinations (series) that converge in the norm of the space. In a Banach space $X$, a sequence $\{e_n\}$ is a Schauder basis if for every $x\in X$, there exists a unique sequence of scalars $\{a_n\}$ such that $x={\sum }_{n=1}^{\infty }{a}_n{e}_n$.

2. Normed and Banach Spaces

A normed vector space $(X,\Vert \cdot \Vert )$ is a vector space $X$ equipped with a norm. The norm induces a metric $d(x,y)=\Vert x-y\Vert$, allowing us to discuss convergence and continuity.

Completeness and Banach Spaces

A normed space is called a Banach Space if it is complete. That is, every Cauchy sequence $\{x_n\}\subset X$ converges to an element $x\in X$.

Fundamental examples include:

1. $C[a,b]$: The space of continuous functions $f:[a,b]\to \mathbb{R}$ with the uniform norm $\Vert f{\Vert }_{\infty }={\sup }_{x\in [a,b]}|f(x)|$.

2. $L^p$ Spaces: For $1\le p<\infty$, $L^p(\Omega ,\Sigma ,\mu )$ is the space of equivalence classes of measurable functions such that ${\int }_{\Omega }|f{|}^{p}d\mu <\infty$, with norm:

   $\Vert f{\Vert }_p={\left({\int }_{\Omega }|f{|}^{p}d\mu \right)}^{1/p}$

   The case $p=2$ is particularly important as it is the only $L^p$ space that is also a Hilbert space.

3. Inner Product and Hilbert Spaces

A Hilbert Space $H$ is a Banach space where the norm is induced by an inner product $⟨\cdot ,\cdot ⟩$, satisfying $\Vert x\Vert =\sqrt{⟨x,x⟩}$.

Geometric Identities and Inequalities

• Cauchy-Schwarz Inequality: $|⟨x,y⟩|\le \Vert x\Vert \Vert y\Vert$. This is fundamental for defining angles in function spaces.

• Parallelogram Law: $\Vert x+y{\Vert }^2+\Vert x-y{\Vert }^2=2(\Vert x{\Vert }^2+\Vert y{\Vert }^2)$.

• Bessel’s Inequality: For any orthonormal set $\{e_n\}$, and any $x\in H$:

  ${\sum }_n|⟨x,e_n⟩{|}^2\le \Vert x{\Vert }^2$

• Parseval’s Identity: If $\{e_n\}$ is a complete orthonormal basis, then equality holds:

  ${\sum }_n|⟨x,e_n⟩{|}^2=\Vert x{\Vert }^2$

4. Bounded Linear Operators

A linear operator $T:X\to Y$ is bounded (and thus continuous) if its operator norm is finite:

$\Vert T\Vert ={\sup }_{x\in X,x\ne 0}\frac{\Vert Tx{\Vert }_Y}{\Vert x{\Vert }_X}$

The space of all bounded linear operators $\mathcal{B}(X,Y)$ is a Banach space if $Y$ is a Banach space.

5. Dual Spaces and Riesz Representation

The Dual Space $X^{\ast }$ (or $X^{\prime }$) consists of all bounded linear functionals $f:X\to \mathbb{K}$.

Riesz Representation Theorem

One of the most profound results in functional analysis states that every continuous linear functional $\varphi$ on a Hilbert space $H$ can be represented as an inner product with a fixed vector $y_{\varphi }\in H$:

$\forall x\in H,\varphi (x)=⟨x,y_{\varphi }⟩$

where $\Vert \varphi {\Vert }_{H^{\ast }}=\Vert y_{\varphi }{\Vert }_H$. This implies that a Hilbert space is isometrically anti-isomorphic to its own dual.

6. Applications: The Framework of Quantum Mechanics

In the Hilbert space formulation of Quantum Mechanics (pioneered by von Neumann), the physical state of a particle is a vector $|\psi ⟩$ in a complex Hilbert space $L^2({\mathbb{R}}^3)$.

• Observables: Physical quantities like energy (Hamiltonian $H$) or momentum ($P$) are represented by Self-Adjoint Operators.
• Eigenvalues: The set of possible measurement outcomes corresponds to the operator’s spectrum.
• Uncertainty: The fact that position $\hat{x}$ and momentum $\hat{p}$ do not commute ($[\hat{x},\hat{p}]=i\hslash$) leads directly to the Heisenberg Uncertainty Principle.

7. Python Implementation: Function Projection and Approximation

We can approximate a function by projecting it onto a finite-dimensional subspace spanned by an orthonormal basis. Here, we use Legendre Polynomials as an orthonormal basis for $L^2[-1,1]$.

import numpy as np
from scipy.integrate import quad
from scipy.special import legendre

def inner_product(f1, f2):
    \"\"\"Calculates the L2 inner product on [-1, 1]\"\"\"
    return quad(lambda x: f1(x) * f2(x), -1, 1)[0]

def project_onto_legendre(f, degree):
    \"\"\"Projects f onto the first 'degree' Legendre polynomials.\"\"\"
    coefficients = []
    
    # We define the projection as a sum of c_n * P_n(x)
    # where c_n = <f, P_n> / <P_n, P_n>
    def projection_func(x):
        total = 0
        for n in range(degree + 1):
            Pn = legendre(n)
            # Normalization constant for P_n on [-1, 1] is 2/(2n+1)
            norm_sq = 2.0 / (2*n + 1)
            
            # Fourier coefficient calculation
            integrand = lambda t: f(t) * Pn(t)
            coeff = quad(integrand, -1, 1)[0] / norm_sq
            
            if n == 0: coefficients.append(coeff) # Store for later check
            total += coeff * Pn(x)
        return total
        
    return projection_func

# Target function: f(x) = exp(x)
f_target = lambda x: np.exp(x)

# Project onto degree 3 Legendre polynomials
proj = project_onto_legendre(f_target, 3)

# Calculate L2 error: ||f - proj||_2
error_integrand = lambda x: (f_target(x) - proj(x))**2
l2_error = np.sqrt(quad(error_integrand, -1, 1)[0])

print(f"Approximating exp(x) on [-1, 1] with degree 3 Legendre polynomials...")
print(f"L2 Error: {l2_error:.6f}")
print(f"Value at x=0.5: Target={np.exp(0.5):.5f}, Proj={proj(0.5):.5f}")

(Note: This lesson provides a rigorous overview. For deeper study, refer to Rudin’s ‘Functional Analysis’ or Kreyszig’s ‘Introductory Functional Analysis with Applications’.)