Tensors and Multilinear Algebra

Tensors provide the ultimate generalization of scalars, vectors, and linear operators. While elementary linear algebra focuses on maps between two vector spaces, multilinear algebra treats functions that depend linearly on multiple variables simultaneously. This framework is essential for general relativity, continuum mechanics, and quantum information theory.

1. Multilinear Maps

Let $V_1,V_2,\dots ,V_k$ and $W$ be vector spaces over a field $\mathbb{F}$. A map $f:V_1\times V_2\times \cdots \times V_k\to W$ is multilinear if it satisfies the linearity condition in each slot independently. For any $v_i,v_i^{\prime }\in V_i$ and $a,b\in \mathbb{F}$:

$f(v_1,\dots ,a{v}_i+b{v}_i^{\prime },\dots ,v_k)=af(v_1,\dots ,v_i,\dots ,v_k)+bf(v_1,\dots ,v_i^{\prime },\dots ,v_k)$

The set of all such multilinear maps forms a vector space, denoted $\mathcal{L}(V_1,\dots ,V_k;W)$. When $W=\mathbb{F}$, these are called multilinear forms.

2. Formal Construction of the Tensor Product

The tensor product $V\otimes W$ is the “friendliest” space that can represent all bilinear maps from $V\times W$. Its construction ensures that multilinear maps can be studied purely through linear tools.

The Free Vector Space

Consider the Cartesian product $V\times W$. We construct the free vector space $F(V,W)$, which consists of all finite linear combinations of ordered pairs $(v,w)\in V\times W$. This space is massive—it treats every pair $(v,w)$ as a linearly independent basis vector.

Quotienting by Bilinear Relations

To enforce linearity, we define a subspace $R\subset F(V,W)$ spanned by the following relations for all $v,v^{\prime }\in V$, $w,w^{\prime }\in W$, and $c\in \mathbb{F}$:

1. $(v+v^{\prime },w)-(v,w)-(v^{\prime },w)$
2. $(v,w+w^{\prime })-(v,w)-(v,w^{\prime })$
3. $(cv,w)-c(v,w)$
4. $(v,cw)-c(v,w)$

The tensor product is the quotient space: $V\otimes W=F(V,W)/R$ The equivalence class of $(v,w)$ in this quotient is denoted by the tensor product symbol: $v\otimes w$.

3. The Universal Property

The utility of the tensor product is captured by its Universal Property: For any bilinear map $B:V\times W\to X$, there exists a unique linear map $L:V\otimes W\to X$ such that the following diagram commutes:

In other words, $B(v,w)=L(v\otimes w)$. This property allows us to replace the non-linear “bilinear” nature of $B$ with the strictly linear map $L$.

4. Tensor Components and (r, s) Notation

A tensor of type $(r,s)$ is an element of the tensor product of $r$ copies of $V$ and $s$ copies of the dual space $V^{\ast }$: $T\in \underset{r\text{ times}}{\underbrace{V\otimes \cdots \otimes V}}\otimes \underset{s\text{ times}}{\underbrace{V^{\ast }\otimes \cdots \otimes V^{\ast }}}$

Using the Einstein summation convention, where repeated indices (one upper, one lower) imply summation, a tensor is expressed in components relative to a basis $\{e_i\}$ and its dual $\{{ϵ}^j\}$: $T=T_{j_1\dots j_s}^{i_1\dots i_r}{e}_{i_1}\otimes \cdots \otimes e_{i_r}\otimes {ϵ}^{j_1}\otimes \cdots \otimes {ϵ}^{j_s}$

• Contravariant indices ($i_r$): Upper indices corresponding to the vector space $V$.
• Covariant indices ($j_s$): Lower indices corresponding to the dual space $V^{\ast }$.

5. Change of Basis and Transformation Laws

How does the component array $T_j^i$ change when we pick a new basis? If $e_{i^{\prime }}=A_{i^{\prime }}^i{e}_i$ is the transition matrix (with inverse $(A^{-1}{)}_i^{i^{\prime }}$), then:

1. Contravariant components (vectors) transform with the inverse matrix: $V^{i^{\prime }}=(A^{-1}{)}_i^{i^{\prime }}{V}^i$
2. Covariant components (covectors) transform with the matrix: ${\omega }_{j^{\prime }}=A_{j^{\prime }}^j{\omega }_j$

An $(r,s)$ tensor transforms as the product of $r$ inverse transformations and $s$ forward transformations.

6. The Metric Tensor: Index Gymnastics

On an inner product space, the metric tensor $g$ is a symmetric $(0,2)$ tensor that defines distances and angles: $g=g_{ij}{ϵ}^i\otimes {ϵ}^j,g_{ij}=⟨e_i,e_j⟩$

The metric provides a canonical isomorphism between $V$ and $V^{\ast }$, allowing us to “raise” or “lower” indices.

• Lowering: Given $V^j$, the covariant version is $V_i=g_{ij}{V}^j$.
• Raising: Given ${\omega }_j$, the contravariant version is ${\omega }^i=g^{ij}{\omega }_j$, where $g^{ij}$ is the matrix inverse of $g_{ij}$.

7. Symmetry, Antisymmetry, and the Wedge Product

Tensors can possess symmetry properties under the permutation of their arguments.

• Symmetric Tensors: $T(v,w)=T(w,v)$. These form the symmetric algebra $S(V)$.
• Alternating Tensors: $T(v,w)=-T(w,v)$. These form the exterior algebra $\Lambda (V)$.

The wedge product ($\wedge$) creates an alternating tensor from two vectors: $v\wedge w=v\otimes w-w\otimes v$ This is the foundation of differential forms and integration on manifolds.

8. Tensor Contraction

Contraction is the operation of summing over an upper and lower index pair. For a $(1,1)$ tensor $T_j^i$, the contraction is simply the trace: $C(T)=T_i^i={\sum }_i{T}_i^i$ This operation reduces a tensor of type $(r,s)$ to type $(r-1,s-1)$. Contraction is basis-independent and represents a “projection” of the tensor’s multilinear capacity.

Python: Efficient Tensor Operations with einsum

The numpy.einsum function allows for a readable and highly efficient implementation of the Einstein summation convention for tensor products and contractions.

import numpy as np

# 1. Setup: Metric tensor (Euclidean) and random tensors
g = np.eye(3) # g_ij
A = np.random.rand(3, 3) # A^ij (Contravariant rank 2)
B = np.random.rand(3, 3) # B_jk (Covariant rank 2)

# 2. Matrix Multiplication via Einstein Summation: C^i_k = A^ij B_jk
# 'ij,jk->ik' means we sum over 'j'
C = np.einsum('ij,jk->ik', A, B)

# 3. Contraction (Trace): T = A^ii
trace_A = np.einsum('ii', A)

# 4. Raising an index: B^ik = g^ij B_jk
g_inv = np.linalg.inv(g) # g^ij
B_raised = np.einsum('ij,jk->ik', g_inv, B)

# 5. Inner product: <u, v> = g_ij u^i v^j
u = np.array([1, 0, 1])
v = np.array([0, 2, 1])
inner_prod = np.einsum('ij,i,j', g, u, v)

print(f"Matrix Product (rank 2):\n{C}")
print(f"Trace: {trace_A:.4f}")
print(f"Inner Product: {inner_prod}")