Obsidian + 🪴 Quartz 4.0

SC - Lezione 2 - Vector-Vector operations, Matrix-Vector operations and their interpretations, Transposition of a product, Matrix-Matrix operations

10 min read

SC - Lezione 2

Checklist

Domande, Keyword e Vocabulary

  • Code to explicitily calculate the sum of two vectors x and y
  • saxpi operation
  • Scalar Product
  • First introduction of the rank of matrix
  • Rank as indices of information of a matrix
  • Sparsity Index
  • Matrix Vector multiplication
  • matrix vector product interpretations
  • Matrix Matrix multiplication
  • Matrix-matrix multiplication interpretations
  • Esempio trasformazione lineare di vettori.

Appunti SC - Lezione 2

Operations on vectors and matrices in Matlab

When considering operations, we have columns and row vectors. The difference between the two is important:

  • In the scalar product, the first vector is a row vector and the second a column vector
  • In the outer product we have a column vector by a row vector

Fundamental operations in Matlab

The first operation is just compute a multiple of a vector It’s done using the * operator.

The sum of two vector is done with + operator and so on.

Code to explicitily calculate the sum of two vectors x and y

Saxpi operation

saxpi operation: the sum of a vector to a multiple of another vector: where is a number. It’s often implemented in old hardware. It’s called saxpi and many operations can be expressed in terms of saxpi operation.

Scalar or inner product in Matlab

The fundamental operation is the scalar product. We will spend a lot of time on this.

Also called inner product between two vectors. The result of such operation is a scalar, that is, a number.

p=x'*y

imagine that x is a column so transposed becomes a row vector. It gives you a number this is why it is called scalar.

Properties of the inner product:

  • Commutativity: but remember that the first vector must be a row vector. p=y'*x
  • Linearity: In other words you can transpose the operations.

We also checked it in the code in matlab.

Manually computing the inner product in Matlab

The inner product can be calculated in Matlab with dot() function or direct computed by:

You can do the componentwise product between two vectors. TO do that you can do in matlab as:

q = sum(x.*y)

Outer product

Is another product or external product, between two column vectors

If x is in , and y is in then the result of the outer product is a matrix in .

Structure of matrix and other operations

A first introduction of rank 1 matrix

What does it mean that a matrix has rank 1? It has very low information. If it does have maximum rank (equal to the number of rows or columns of the matrix), then it contains the maximum information possible.

ChatGPT 4.0

ChatGPT 4.0, respect to the previous version, introduces the approximation of matrix with row rank.

A possible application of a rank 1 matrix is that you could compress it into only one column. Say for example that we have a matrix 10 by 10 with 100 components. If it has rank 1, it can be reduced to an outer product, so the information to store is just 20 numbers.

We will return on this very deeply.

Other functions and features

  • Random function and rand integer function
  • Matrix extraction: diagonal, upper and bottom triangular
  • Notation for extracting Matrices
A(:,1) % First column
A(1,:) % First row
A(1:3,:) %First tree rows only, all the columns
A(:,2:end) %The 2 to the last column

Matrix - Vector multiplication

It's very important to understand the various interpretation of the matrix vector multiplication

First interpretation

The first component of the result is the scalar product of the first row of the matrix with the vector, and this pattern continues for each row. In other words, each component of the result is obtained by multiplying the corresponding row of the matrix with the vector.

  • The vector must have number of elements equal to the number of columns of the matrix A
  • The resulting vector will have the same number of rows of the matrix A
  • The vector must have the same number of elements of the subvector of the matrix obtainted by considering each row singularly. In other words, if the matrix has 5 elements on each row, must have 5 elements (even if it’s a column vector).

Matrix Vector application of the first interpration: ChatGPT

For example, ChatGPT works by utilizing thousands of matrices, and its fundamental computation involves matrix-by-matrix multiplication. In ChatGPT, any work is a vector. The designer determines the length of this vector, for instance ChatGPT 4 has around 12k vectors to represent a single word.

The word you input are then associated with all the words on the dataset.

Imagine a matrix with 10 vectors representing 10 words, and each vector is an embedding. To determine which word is the most similar to a given input, matrix multiplication is performed, as it efficiently measures similarity.

If the result of a single number in the vector is like zero, then they are orthogonal they don’t share any information. They can also be negative or positive.

Matrix vector multiplication in terms of scalar product

Second interpretation

This interpretation consider the inner product as linear combination of the columns of the matrix A

Each column of A is multiplied by the corresponding component of the vector v and are then added together. It uses the saxpi operation we saw before

z = zeros(m,1);
for j=1:n 
	z = z+ A(:,j)*v(j);
end

Note

A(:,j)*v(j) is an outer product, the result is a vector. In this way we obtain vectors to be summed togheter.

The following code is just a clarification of the previous concept.

z = zeros(m,1)
partial_sum= zeros(m,n)
for j=1:n 
    ps(:,j) =A(:,j)*v(j) 
end
disp(ps) % you obtain a 5x3 matrix of partials sum
 
for j=1:n
    z = z + ps(:,j);
end

Product interpretation variant: transposition

When you transpose a product you have to transpose both factors and interchange the order of factors them so becomes

ChatGPT

Sometimes on chatgpt they interpret the chatgpt in terms of vector matrix multiplication

Matrix vector multiplication is a combination of rows and columns. If we want to interpret this transpose it is the column

zT=zeros(1,n);
for i=1:m 
	zT=zT + A(i,:)*v(i);
end
zT

Matrix - Matrix multiplication

First Interpretation

C=A*B

First row scalar product by first column we have the first column of the result matrix

The scalar product of the first row with the first column gives the first element of the first column of the result matrix. The scalar product between the second row of A and the first column B gives the second element of the first column of C. The scalar product between the first row of A and the second column of B gives the first element of the second column of C.

Code example

A code that shows this:

A = [2 -1 3; 4 -1 1; 0 6 5; 6 1 0]
B = [-1 2; 7 2; -3 1]
[m,n]=size(A);
[n,k]=size(B);
C= A*B
 
A(1,:)* B(:,1)
A(2,:)*B(:,1)
A(3,:)*B(:,1)
A(4,:)*B(:,1)
 
A(1,:)* B(:,2)
A(2,:)*B(:,2)
A(3,:)*B(:,2)
A(4,:)*B(:,2)
 
 
 
C = zeros(m,k);
for j=1:k
    for i=1:m
        C(i,j)=A(i,:) * B(:,j);
    end
end
disp(C)

Interpretation using the matrix - vector product

the matrix - matrix product is interpretable using the matrix - vector product operation.

The j-th column of C is the matrix - vector product of A and the j-th column of B.

C = zeros(m,p);
for j=1:p
    C(:,j) = A*B(:,j);
end
C

Interpretation with the Outer Product

The outer product is the last interpration. The matrix C is the sum between all the matrices obtainted by the outer product of the j-column of A by the j-row of B

Going by the picture. If i consider the first coumn and the first row i get the outer product and i get a matrix with the same rows and columns of the output matrix. Then i consider the second column, second row and i get another matrix rank 1. By iterating, I got N matrices as many as the number of columns and all those matrix are rank 1 matrix so they have very low information. If i do the summation of the matrix i get the result.

Matlab Example

A = [2 -1 3; 4 -1 1; 0 6 5; 6 1 0]
B = [-1 2; 7 2; -3 1]
[m,n]=size(A);
[n,k]=size(B);
AB = zeros(m,k,n);
C= zeros(m,k);
% Compute and store each outer product
for j = 1:n
    AB(:,:,j) = A(:,j) * B(j,:);  % Outer product of j-th column of A and j-th row of B
end
 
% Sum the partial matrices to get the final result
for j = 1:n
    C = C + AB(:,:,j);  % Accumulate the outer products
end
C
C2=A*B

This matlab code shows exatly this.

Rank 1 matrices

for j = 1:n
    rank(AB(:,:,j))
end

I got matrices as many as the number of columns and all those matrix are rank 1 matrix so they have very low information.

This fact should make us to think that if i get a matrix, i can write this matrix as a rank 1 matrix. It’s like decomposing a matrix of maximum rank by n matrices with low rank, that holds less informations. This in future will be used in some applications like SVD.

Example of linear transformation of vectors

We want to rotate this polygon: Like this:

To do that, (as we will see in the next lessons), we need an orthogonal matrix that rotates these pointes.

We first translate the origin of the reference system into the centroid of the polygon:

xc = mean(x(1:end-1)); yc = mean(y(1:end-1)); % last vertex is fictitious and must not be used in the computation of the averages
xx = x - xc; yy = y - yc;

We consider the following rotation matrix:

and we choose an arbitrary

Then we do the following product:

Star = [xx ; yy]; 
R =[cos(theta) -sin(theta); sin(theta) cos(theta)]; 
R*Star

where is the rotation matrix and is the matrix composed of the two vectors translated to the centroid.

Why the product? Recall the second interpretation of a matrix-matrix multiplication. I’m doing the product of the entire rotation matrix by the columns of . Recall that (Star) is a matrix composed of two vector, where each column is the couple of coordinates.

In other words, i’m applying the rotation to each couple of coordinates.

The result of this product is the following:

Graph View

Backlinks