Spectral Theory: Eigenvalues, Eigenvectors, and Diagonalization

Spectral theory is the study of linear operators through their invariant subspaces and characteristic values. In the context of finite-dimensional vector spaces, this reduces to the analysis of the structure of square matrices.

1. The Eigenvalue Equation

Let $V$ be a vector space over a field $\mathbb{F}$ (typically $\mathbb{R}$ or $\mathbb{C}$), and let $T:V\to V$ be a linear operator. A scalar $\lambda \in \mathbb{F}$ is called an eigenvalue of $T$ if there exists a non-zero vector $\mathbf{v}\in V$, called an eigenvector, such that: $T(\mathbf{v})=\lambda \mathbf{v}$

In matrix form, if $A\in {\mathbb{F}}^{n\times n}$ represents $T$ with respect to some basis, the equation becomes: $A\mathbf{v}=\lambda \mathbf{v}⟹(A-\lambda I)\mathbf{v}=0$

where $I$ is the identity matrix. The eigenvector $\mathbf{v}$ must be non-zero because $A0=\lambda 0$ is always true for any $\lambda$ and provides no information about $A$.

2. The Characteristic Polynomial and the Spectrum

The condition $(A-\lambda I)\mathbf{v}=0$ for $\mathbf{v}\ne 0$ implies that the operator $(A-\lambda I)$ is not injective, which for finite-dimensional spaces is equivalent to saying the matrix $(A-\lambda I)$ is singular. Thus, its determinant must vanish: $p(\lambda )=\det (A-\lambda I)=0$

$p(\lambda )$ is a polynomial in $\lambda$ of degree $n$, known as the characteristic polynomial. The set of all eigenvalues of $A$ is called the spectrum of $A$, denoted by $\sigma (A)$: $\sigma (A)=\{\lambda \in \mathbb{C}\mid \det (A-\lambda I)=0\}$

By the Fundamental Theorem of Algebra, $p(\lambda )$ has exactly $n$ complex roots (counting multiplicity).

3. Algebraic vs. Geometric Multiplicity

When analyzing the roots of $p(\lambda )$, we distinguish between two types of multiplicity for an eigenvalue ${\lambda }_i$:

1. Algebraic Multiplicity ($AM({\lambda }_i)$): The number of times ${\lambda }_i$ appears as a root of the characteristic polynomial. If $p(\lambda )=(\lambda -{\lambda }_i{)}^{k}q(\lambda )$ where $q({\lambda }_i)\ne 0$, then $AM({\lambda }_i)=k$.
2. Geometric Multiplicity ($GM({\lambda }_i)$): The dimension of the eigenspace $E_{{\lambda }_i}=\text{null}(A-{\lambda }_{i}I)$. It represents the number of linearly independent eigenvectors associated with ${\lambda }_i$.

Theorem: For any eigenvalue $\lambda$, $1\le GM(\lambda )\le AM(\lambda )$. If $GM(\lambda )<AM(\lambda )$, the eigenvalue is said to be defective. A matrix is defective if it has at least one defective eigenvalue.

4. Similarity Transformations

Two matrices $A$ and $B$ are similar ($A\sim B$) if there exists an invertible matrix $P$ such that: $B=P^{-1}AP$

Similarity is an equivalence relation that represents the same linear operator under different bases. Theorem: Similar matrices share the same characteristic polynomial, and thus the same eigenvalues, trace, and determinant. Proof sketch: $\det (B-\lambda I)=\det (P^{-1}AP-\lambda P^{-1}IP)=\det (P^{-1}(A-\lambda I)P)=\det (P^{-1})\det (A-\lambda I)\det (P)=\det (A-\lambda I)$

5. Diagonalizability

A matrix $A\in {\mathbb{F}}^{n\times n}$ is diagonalizable if it is similar to a diagonal matrix $D$. That is, $A=PD{P}^{-1}$.

Condition for Diagonalizability: $A$ is diagonalizable if and only if it possesses $n$ linearly independent eigenvectors. This occurs if and only if for every eigenvalue ${\lambda }_i$, the geometric multiplicity equals the algebraic multiplicity ($GM=AM$).

If $A$ is diagonalizable, the columns of $P$ are the eigenvectors of $A$, and the diagonal entries of $D$ are the corresponding eigenvalues: $D=\left(\begin{array}{cccc}{\lambda }_1& 0& \dots & 0\\ 0& {\lambda }_2& \dots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \dots & {\lambda }_n\end{array}\right)$

6. The Cayley-Hamilton Theorem

The Cayley-Hamilton Theorem states that every square matrix satisfies its own characteristic equation. If $p(\lambda )=\det (A-\lambda I)=(-1{)}^n{\lambda }^n+c_{n-1}{\lambda }^{n-1}+\cdots +c_1\lambda +c_0$, then: $p(A)=(-1{)}^n{A}^n+c_{n-1}{A}^{n-1}+\cdots +c_{1}A+c_{0}I=0$

This theorem is powerful for computing:

• Matrix Powers: $A^n$ can be expressed as a linear combination of lower powers $\{I,A,\dots ,A^{n-1}\}$.
• Inverses: If $A$ is invertible ($c_0\ne 0$), then $A^{-1}=-\frac{1}{c_0}((-1{)}^n{A}^{n-1}+\cdots +c_{1}I)$.

7. Numerical implementation in Python

We can verify the diagonalization property $A=PD{P}^{-1}$ using numpy.

import numpy as np

# Define a non-defective matrix
A = np.array([[4, -2,  1],
              [1,  1,  0],
              [0,  0,  2]])

# Compute eigenvalues and eigenvectors
# evals: eigenvalues, evecs: matrix where columns are eigenvectors
evals, evecs = np.linalg.eig(A)

print("Eigenvalues:", evals)
print("Eigenvectors matrix P:\n", evecs)

# Create diagonal matrix D
D = np.diag(evals)

# Verify A = P D P^-1
# Using np.allclose to handle floating point precision
P = evecs
P_inv = np.linalg.inv(P)
A_reconstructed = P @ D @ P_inv

print("\nReconstructed A:\n", A_reconstructed)
print("\nVerification (A == PDP^-1):", np.allclose(A, A_reconstructed))

# Verify Characteristic Polynomial via Cayley-Hamilton
# p(lambda) = det(A - lambda*I)
# For this 3x3, let's find coefficients using np.poly
coeffs = np.poly(A) 
# coeffs[0]*A^3 + coeffs[1]*A^2 + coeffs[2]*A + coeffs[3]*I
A_sq = A @ A
A_cub = A_sq @ A
CH_check = coeffs[0]*A_cub + coeffs[1]*A_sq + coeffs[2]*A + coeffs[3]*np.eye(3)
print("\nCayley-Hamilton check (should be ~0):\n", np.round(CH_check, 10))

8. Physical and Practical Relevance

Stability Analysis

In differential equations, a system $\dot{\mathbf{x}}=A\mathbf{x}$ is stable if all eigenvalues of $A$ have negative real parts. The eigenvectors define the “modes” of the system—decoupled directions along which the system evolves independently.

Vibration and Resonance

In mechanical engineering, the eigenvalues of a mass-stiffness matrix correspond to the natural frequencies of a structure. The eigenvectors are the mode shapes, describing how the structure deforms at those frequencies.

Information Retrieval: PageRank

The PageRank algorithm treats the internet as a massive graph and finds the principal eigenvector (the one corresponding to $\lambda =1$) of a modified adjacency matrix. This eigenvector represents the stationary distribution of a random walk, highlighting important nodes.

Knowledge Check