Vectors
A vector is a mathematical object that has two distinct characteristics: magnitude (size or length) and direction.
It can be written as
From a geometric point of view, it can be seen as an arrow:
- the length of the arrow represents the magnitude
- the orientation of the arrow represents the direction
Vectors are the building blocks of matrices.
The key operations are:
- The addition:
- The dot product:
- The scalar product:
generalized to dimensions of a vector it can be written as: - Normalization: if we compute the norm of a vector
then we can normalize a vector by doing: so that its length is exactly 1, creating a unit vector. The intuition is that the length of a vector is computed as .
Basis
A basis is a set of vectors:
Linear Combinations:
Any vector
This means there is only one set of scalars (coordinates)
Types of basis:
- Standard Basis:
and . For example means 3 steps right and 2 steps up. - Alternative Basis: If we change the basis (e.g., rotate it or stretch it), the same physical vector will have a different list of coordinates.
Some example of alternative basis are:
- eigenbasis using Singular Value Decomposition (i.e. Principal Component Analysis)
- frequency basis when we use Fourier analysis
- orthonormal basis
Orthogonal basis It holds the following property:
- Orthogonality:
Orthornormal basis Intuition: every vector in the set must be at right angle (90°) to every other vector and every vector must have length (magnitude) of exactly 1. Formally, the following two property holds:
- Orthogonality:
- Normalized:
In a compact notation one could use the kronecker delta (
- where
if , and if .
An example of orthonormal basis is the standard basis for
If you take the dot product of any two, you get