In the previous post we saw how matrices/vectors interact with scalar arithmetic. E.G. adding a fixed value (scalar) to a matric/vector. There is also a set of arithmetic functions involving vector to vector or matric to matrix operatons. The one I vaguely remember from a linear algrebra class is the dot product (a form of matrix multiplication). There are also other operations that may or may not be useful as we continue, but I will mention them just in case.

The one thing to keep in mind for all these operations is that the shapes of the matrices must match in some fashion or other.

Add/Subtract Two Matrices

Let’s start with something simple. We’ll look at vectors then matrices. A pattern that will continue throughout this post.

In [1]:
import numpy as np
In [2]:
# create some vectors and matrices
v1 = np.arange(1, 4)
v2 = np.array([2, 4, 6])
m1 = np.arange(1, 7).reshape(3,2)
m2 = m1 + np.arange(1, 3)
print('v1:', v1, '\nv2:', v2, '\n\nm1:\n', m1, '\n\nm2:\n', m2)

v1: [1 2 3] 
v2: [2 4 6] 

m1:
[[1 2]
[3 4]
[5 6]]


m2:
[[2 4]
[4 6]
[6 8]]

In [3]:
# let's try adding and subtracting the vectors
print(f"v1 + v2: {v1 + v2}")
print(f"v1 - v2: {v1 - v2}")
print(f"v2 - v1: {v2 - v1}")

v1 + v2: [3 6 9]
v1 - v2: [-1 -2 -3]
v2 - v1: [1 2 3]
In [4]:
# and now the matrices
print(f"m1 + m2:\n{m1 + m2}\n")
print(f"m1 - m2:\n{m1 - m2}\n")
print(f"m2 - m1:\n{m2 - m1}\n")

m1 + m2:
[[ 3  6]
 [ 7 10]
 [11 14]]

m1 - m2:
[[-1 -2]
[-1 -2]
[-1 -2]]


m2 - m1:
[[1 2]
[1 2]
[1 2]]

In [5]:
# now what about adding a vector to a matrix
print(f"m2 + v1:\n{m2 + v1}")

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-70b64ec82970> in <module>
      1 # now what about adding a vector to a matrix
----> 2 print(f"m2 + v1:\n{m2 + v1}")
ValueError: operands could not be broadcast together with shapes (3,2) (3,)

Broadcast?? We’ll get to that later. Though you’ve already seen it in operation.

Multiplication

Now matrix multiplication has a couple of twists and turns. We’ve already looked at mutliplication by a scalar, so we will ignore that option.

Hadamard Product

If we use the Python * multiplication operator you will get pretty much what you expect. This is known as the Hadamard product.

In [6]:
print(f"v1 * v2 = {v1} * {v2} = {v1 * v2}")
print(f"\nm1 * m2 =")
m1m2 = m1 * m2
print(f"[{m1[0]}    [{m2[0]}    [{m1m2[0]}")
print(f" {m1[1]}  *  {m2[1]}  =  {m1m2[1]}")
print(f" {m1[2]}]    {m2[2]}]    {m1m2[2]}]")
print(f"\nm1 * v2 = ", end='')
print(m1 * v2)

v1 * v2 = [1 2 3] * [2 4 6] = [ 2  8 18]

m1 * m2 =
[[1 2]    [[2 4]    [[2 8]
[3 4]  *  [4 6]  =  [12 24]
[5 6]]    [6 8]]    [30 48]]


m1 * v2 = 

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-fec9d89825c1> in <module>
      6 print(f" {m1[2]}]    {m2[2]}]    {m1m2[2]}]")
      7 print(f"\nm1 * v2 = ", end='')
----> 8 print(m1 * v2)

ValueError: operands could not be broadcast together with shapes (3,2) (3,)

Dot Product

However that is probably not he most common product used in linear algebra. That would likely be the dot product. I am going to ignore, at least for now, the cross product, etc.

Wasn’t going to discuss the dot product very much, but decided to have, at least, a little look at it. In the case of two vectors — of equal size — their dot product will be a scalar. For two matrices — of suitable shapes — you will get another matrix. The value in each element of the result will be the dot product of two vectors drawn from the original two matrices.

For two vectors with n elements, the dot product is:

$$a \bullet b = \sum_{i=1}^n a_{i}b_{i}$$
In [7]:
# let's start with our vectors
v1v2 = np.dot(v1, v2)
print(f"v1 dot v2 = v1[0]*v2[0] + v1[1]*v2[1] + v1[2]*v2[2] = ", end='')
print(f"{v1[0]*v2[0]} + {v1[1]*v2[1]} + {v1[2]*v2[2]} = {v1v2}")

v1 dot v2 = v1[0]*v2[0] + v1[1]*v2[1] + v1[2]*v2[2] = 2 + 8 + 18 = 28

Given two matrices, a with n columns and m rows and b with p columns and n rows, the result for each element in the dot product is given by:

$$c_{ij} = \sum_{k=1}^n a_{ik}b_{kj}$$

where: i = 1, …, m and
j = 1, …, p

And, the output array will have m rows and p columns.

The dot product of the 2 matrices is usually written as ab or ba. Basically, we treat the rows of a and the columns of b as vectors and calculate their dot product to get the entries for the result matrix. Which means that the number of columns in a must equal the number of rows in b. And, the resulting matrix can have an entirely different shape from either of the two initial matrices. For example, given a 3 x 2 matrix A and a 2 x 4 matrix B, AB will produce a 3 x 4 matrix result. And, BA is not defined as in that orientation the number of columns in B doesn’t match the number of rows in A.

In [ ]:
# now let's try our matrices
m1m2 = np.dot(m1, m2)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-8-2ffd071fa820> in <module>
      1 # now let's try our matrices
----> 2 m1m2 = np.dot(m1, m2)

<array_function internals> in dot(*args, **kwargs)


ValueError: shapes (3,2) and (3,2) not aligned: 2 (dim 1) != 3 (dim 0)