Calculus of Variations: The Geometry of Functional Optimization

The Calculus of Variations (CoV) represents a transition from the optimization of finite-dimensional vectors to the optimization of infinite-dimensional objects—functions. While standard differential calculus identifies critical points $x\in {\mathbb{R}}^n$ of a function $f(x)$, CoV identifies “critical functions” $y(x)$ that extremize a functional $J[y]$. This field is the mathematical foundation of Lagrangian mechanics, general relativity, and minimal surface theory.

1. Functionals and the Domain of Optimization

A functional $J$ is a mapping from a function space $\mathcal{Y}$ (typically a Sobolev space or $C^k$ space) to the real numbers. The canonical form encountered in physics and geometry is the integral functional:

$J[y]={\int }_{x_1}^{x_2}L(x,y(x),y^{\prime }(x))dx$

Here, $L(x,y,y^{\prime })$ is the Lagrangian. The domain is usually restricted by Dirichlet boundary conditions: $y(x_1)=y_1$ and $y(x_2)=y_2$. The goal is to find $y\in \mathcal{Y}$ such that $J[y]$ is a local extremum.

2. The First Variation and Stationary Conditions

To find an extremum, we consider a variation $y(x)\to y(x)+ϵ\eta (x)$, where $ϵ$ is a small parameter and $\eta (x)$ is a smooth “test function” satisfying $\eta (x_1)=\eta (x_2)=0$. The variation of the functional is defined as the Gâteaux derivative:

$\delta J[y;\eta ]={\frac{d}{dϵ}J[y+ϵ\eta ]|}_{ϵ=0}$

For $y$ to be a stationary point, we require $\delta J=0$ for all admissible $\eta$. Using the chain rule:

$\delta J={\int }_{x_1}^{x_2}\left[\frac{\partial L}{\partial y}\eta (x)+\frac{\partial L}{\partial y^{\prime }}{\eta }^{\prime }(x)\right]dx=0$

Applying integration by parts to the second term:

${\int }_{x_1}^{x_2}\frac{\partial L}{\partial y^{\prime }}{\eta }^{\prime }dx={\left[\frac{\partial L}{\partial y^{\prime }}\eta \right]}_{x_1}^{x_2}-{\int }_{x_1}^{x_2}\frac{d}{dx}\left(\frac{\partial L}{\partial y^{\prime }}\right)\eta dx$

Since $\eta$ vanishes at the boundaries, the first term disappears. The condition $\delta J=0$ becomes:

${\int }_{x_1}^{x_2}\left(\frac{\partial L}{\partial y}-\frac{d}{dx}\frac{\partial L}{\partial y^{\prime }}\right)\eta (x)dx=0$

3. The Euler-Lagrange Equation

By the Fundamental Lemma of the Calculus of Variations, if the integral of $f(x)\eta (x)$ is zero for every smooth $\eta$ with compact support, then $f(x)$ must be zero. This yields the Euler-Lagrange Equation:

$\frac{\partial L}{\partial y}-\frac{d}{dx}\left(\frac{\partial L}{\partial y^{\prime }}\right)=0$

This second-order differential equation is the necessary condition for $y(x)$ to be an extremizer.

The Beltrami Identity

When the Lagrangian $L$ has no explicit dependence on $x$ ($\partial L/\partial x=0$), the E-L equation admits a first integral:

$L-y^{\prime }\frac{\partial L}{\partial y^{\prime }}=\text{constant}$

In physics, this constant often represents the conservation of energy.

4. Classical Variational Problems

4.1 Geodesics: Shortest Paths

A geodesic is a curve that extremizes the distance functional $S=\int ds$. On a plane, $d{s}^2=d{x}^2+d{y}^2$, so $L=\sqrt{1+(y^{\prime }{)}^2}$. The E-L equation reduces to $\frac{d}{dx}(\frac{y^{\prime }}{\sqrt{1+y^{\prime 2}}})=0$, implying $y^{\prime }$ is constant—a straight line. On curved manifolds, geodesics are governed by the metric tensor $g_{ij}$ and the Christoffel symbols.

4.2 The Brachistochrone

The problem of finding the curve $y(x)$ that minimizes the time of travel for a mass sliding under gravity. Using conservation of energy $v=\sqrt{2gy}$, the time functional is:

$T[y]=\int \frac{\sqrt{1+(y^{\prime }{)}^2}}{\sqrt{2gy}}dx$

Applying the Beltrami Identity to $L=\sqrt{(1+(y^{\prime }{)}^2)/y}$ leads to the differential equation for a cycloid.

5. Constraints and Lagrange Multipliers

In “Isoperimetric” problems, we extremize $J[y]$ subject to a constraint $G[y]=\int M(x,y,y^{\prime })dx=C$. We construct the augmented Lagrangian:

$\bar{L}=L+\lambda M$

where $\lambda$ is a constant Lagrange multiplier. An example is finding the shape of a hanging chain (the Catenary), which minimizes gravitational potential energy subject to a fixed length.

6. From Lagrangian to Hamiltonian Dynamics

The transition to Hamiltonian mechanics involves a Legendre transform. We define the generalized momentum:

$p=\frac{\partial L}{\partial y^{\prime }}$

The Hamiltonian is defined as $H(x,y,p)=p{y}^{\prime }-L$. This transforms the second-order E-L equation into a system of two first-order equations:

$\frac{dy}{dx}=\frac{\partial H}{\partial p},\frac{dp}{dx}=-\frac{\partial H}{\partial y}$

This “canonical” form is central to quantum mechanics and statistical field theory.

7. Noether’s Theorem: Symmetry and Conservation

Noether’s Theorem states that for every continuous symmetry of the action $S=\int Ldt$, there is a corresponding conserved quantity.

Suppose $L$ is invariant under a transformation $y\to y+ϵ\psi$. Then: $\frac{dL}{dϵ}=\frac{\partial L}{\partial y}\psi +\frac{\partial L}{\partial y^{\prime }}{\psi }^{\prime }=0$ Substituting the E-L equation $\frac{\partial L}{\partial y}=\frac{d}{dx}\frac{\partial L}{\partial y^{\prime }}$: $\frac{d}{dx}\left(\frac{\partial L}{\partial y^{\prime }}\right)\psi +\frac{\partial L}{\partial y^{\prime }}{\psi }^{\prime }=\frac{d}{dx}\left(\frac{\partial L}{\partial y^{\prime }}\psi \right)=0$ Thus, $Q=\frac{\partial L}{\partial y^{\prime }}\psi$ is a constant of motion.

8. The Second Variation and Legendre’s Condition

To distinguish between a minimum and a maximum, we look at ${\delta }^{2}J$. A necessary condition for a minimum is Legendre’s Condition:

$\frac{{\partial }^{2}L}{\partial y^{\prime 2}}\ge 0$

If $L_{y^{\prime }{y}^{\prime }}<0$ along the path, the stationary point is a local maximum.

9. Python Implementation: Numerical Shooting Method

Analytic solutions are rare. Here we solve the Brachistochrone ODE $y^{\prime \prime }=-\frac{1+(y^{\prime }{)}^2}{2y}$ using a boundary value problem solver.

import numpy as np
from scipy.integrate import solve_bvp
import matplotlib.pyplot as plt

def brachistochrone_ode(x, y):
    # y[0] is position, y[1] is derivative y'
    # Adding a small epsilon to avoid division by zero at y=0
    return np.vstack((y[1], -(1 + y[1]**2) / (2 * (y[0] + 1e-6))))

def boundary_conditions(ya, yb):
    # Start at height 1.0, end at height 0.2
    return np.array([ya[0] - 1.0, yb[0] - 0.2])

x_nodes = np.linspace(0, 1, 100)
y_initial = np.linspace(1, 0.2, 100).reshape(1, -1)
yp_initial = np.zeros((1, 100))
y_guess = np.vstack((y_initial, yp_initial))

sol = solve_bvp(brachistochrone_ode, boundary_conditions, x_nodes, y_guess)

if sol.success:
    plt.plot(sol.x, sol.y[0], label='Numerical Brachistochrone')
    plt.gca().invert_yaxis()
    plt.legend()
    plt.show()