Tensors and tensor products are multilinear generalizations of vectors and matrices. They provide a flexible framework for representing and manipulating data in various fields. In fact, there are multiple ways to define tensors and tensor products. We will start with an abstract algebraic definition, and then provide constructive definitions in terms of multilinear maps.
Multilinear Map
Suppose
and are vector spaces. A function is called multilinear if it is linear in each variable. That is, when for all , the function is linear for each with . In particular, a bilinear function is a multilinear function of two variables.
Tensor Product of Vector Spaces
Tensor Product of Vector Spaces
The tensor product of vector spaces
and over the same field is a vector space together with a bilinear function such that for every vector space and bilinear function there exists a unique linear function such that : The function
usually stays anonymous and is written as .
Construction of Tensor Product of Vector Spaces
The tensor product of vector spaces
and over the same field is a vector space of the same field spanned by the set of all pure (elementary) tensors of the form for and with the following properties hold:
. . for all .
Intuition of Tensor Products
Tensor products of vector spaces are to Cartesian products of sets as direct sums of vectors spaces are to disjoint unions of sets.
Proposition
A basis of
is given by the set of all pure tensor products of basis vectors of and .
Proof
Proposition
Tensors as Multilinear Maps
Theorem
Suppose
and are finite dimensional vector spaces. The tensor product space of dual space of and can be naturally identified with the space of linear maps . That is
Proof For any
What does it mean by "naturally identified"?
This is a subtle but interesting point. It means that this isomorphism is “special” and “nice”, so that it does not depend on the choice of basis. In fact, this is what covectors are all about. Covectors are designed to eliminate the need to fix a basis when working with dual spaces. By contrast, if the isomorphism did depend on the choice of basis, it would be considered non-canonical, and we would typically only assert the existence of an isomorphism, without identifying a specific one.
Tensor
A tensor
of type over a vector space on the field is a multilinear map: where denotes the dual space of , is called contravariant order and is called covariant order.
e.g. A matrix
Proposition
Tensors of type
over a vector space are exactly elements of the tensor space
Tensor Product of Tensors
Let
and be two tensors of types and respectively. Then the tensor product is the tensor of type defined by for all vectors and covectors .