Vector Spaces

$\gdef \N{\mathbb{N}} \gdef \Z{\mathbb{Z}} \gdef \Q{\mathbb{Q}} \gdef \R{\mathbb{R}} \gdef \C{\mathbb{C}} \gdef \setcomp#1{\overline{#1}} \gdef \sseq{\subseteq} \gdef \pset#1{\mathcal{P}(#1)} \gdef \covariant{\operatorname{Cov}} \gdef \of{\circ} \gdef \p{^{\prime}} \gdef \pp{^{\prime\prime}} \gdef \ppp{^{\prime\prime\prime}} \gdef \pn#1{^{\prime\times{#1}}} $

April 23 Lecture. Links to the board and the recording.

Examples of Vector Spaces¶

Vectors in \(\R^n\) are objects on which we define operations of addition (to each other) and multiplication by a scalar.
Other “objects” on which we can define the same operations:

• Matrices: \(\begin{bmatrix}a&b&c\\d&e&f\end{bmatrix}\)
  • Denoted as \(M_{m\times n}\) with an order of \(m\) rows by \(n\) columns
  • Addition between 2 matrices works by adding corresponding entries into the resulting matrix. You can only add matrices of the same order
    • \(\begin{bmatrix}a&b&c\\d&e&f\end{bmatrix} + \begin{bmatrix}g&h&i\\ j&k&l\end{bmatrix} = \begin{bmatrix}a+g&b+h&c+i\\d+j&e+k&f+l\end{bmatrix}\)
  • Multiplication works by multiplying each term by the scalar
    • \(n\cdot\begin{bmatrix}a&b&c\\d&e&f\end{bmatrix}=\begin{bmatrix}a\cdot n&b\cdot n&c\cdot n\\d\cdot n&e\cdot n&f\cdot n\end{bmatrix}\)
• Polynomials: \(a_{n}x^{n}+a_{n-1}x^{n-1}+a_{n-2}x^{n-2}+\dots+a_1x+a_2\) (“formal sum” of powers of variable \(x\), each with a coefficient)
  • Degree of polynomial is the highest power of \(x\)
  • Addition and scalar multiplication works as expected
• Functions:
  • Let \(f(x)=\sin x\) and let \(g(x)=4e^x\).
  • Addition \((f+g)(x)=\sin x+4e^x\)
  • Multiplication \((c\cdot f)(x)=c\cdot f(x) = c\cdot \sin x\)

All of these are examples of abstract vector spaces.

Definition of Vector Space¶

A set \(V\) is called a vector space (and its elements are called vectors) if we can define on \(V\) operations of addition (\(\oplus\)) and scalar multiplication (\(\odot\)) satisfying the following ten axioms:

Vector Space Axioms¶

Common Vector Spaces:¶

• \(\R^n\)
• \(M_{m\times n}(\R)\) = all matrices of order \(m\times n\) \((m,n\in\N)\) with entries in \(\R\)
• \(\R_n\left[x\right]\) = set of all polynomials in variable \(x\) with real coefficients and of degree \(\le n\).
  • Note: Not degree = \(n\), because that wouldn’t be a vector space

But why are we doing this?¶

We can prove properties and define concepts just based on the axioms, and these properties/concepts will automatically apply ot all the examples of vector spaces.

Properties of Vector Spaces¶

• \(c\odot\vec0=\vec0\)
• \(0\odot\vec v=\vec0\)
• \(-1\odot\vec v=\vec v^\prime\)

April 27 Tirgul. Links to the board and the recording.

Subspaces¶

Definition: Let V be a vector space. A subset \(U\subseteq V\) is a subspace of \(V\) (denoted \(U \le V\)) if \(U\) is itself a vector space under same operations \(\oplus,\odot\) as defined in \(V\).
Examples of subspaces of \(\R^2\):
\(U_1=x\)-axis
\(U_2=y\)-axis
\(U_3=xy\) line
“Trivial” subspaces of \(\R^2\):
1. \({(0,0)}\)
2. All of \(\R^2\)

What are subspaces of \(\R^2\)?¶

1. \(\{(0,0)\}\)
2. All of \(\R^2\)
3. Any line through the origin (\(y=kx\) or \(y\)-axis)
   • Closure under addition:
     • \(\begin{aligned}(x_1,kx_1)+(x_2,kx_2)&=(x_1+x_2,kx_1+kx_2)\\&=(x_1+x_2, k(x_1+x_2))\\&\in\text{ line }y=kx\end{aligned}\)
     • \((0,y_1)+(0,y_2)=(0,y_1+y_2) \in y\)-axis
   • Closure under scalar multiplication:
     • \(\begin{aligned}c\cdot(x, kx)&=\\(cx,ckx)&=\\(cx, k(cx))&\in\text{ line }y=xk\end{aligned}\)
     • \(c\cdot(0, y)=(0,cy)\in y\)-axis

What are subspaces of \(\R^3\)?¶

1. \(\{(0,0,0)\}\)
2. Any line through origin
3. Any plane going through origin
4. All of \(\R^3\)

April 30 Lecture. Links to the board and the recording.
[Incomplete] Coming soon to a PDF near you…

Checking if a subset is a subspace¶

\(U\subseteq V\) is a subspace of V (\(U\le V\)) iff

1. \(U\not=\emptyset\)
2. \(U\) is closed under addition
3. \(U\) is closed under scalar multiplication
   See the board for examples, I can’t be bothered to type them all out.

Intersection and Union of Subspaces¶

Suppose \(U_1,U_2\le V\).
\(U_1\cap U_2\le V\) is always true
\(U_1\cup U_2\le V\) is true ONLY if \(U_1\subseteq U_2\) or \(U_2\subseteq U_1\).

May 4 Tirgul. Links to the board and the recording.

Linear Combination¶

Definition:
Given a finite set \(S=\{\vec v_1,\dots,\vec v_k\}\) in any vector space V, a linear combination of \(S\) is any vector of form \(\vec u=c_1\vec v_1+\dots+c_k\vec v_k\).

Span¶

Definition:
The span of a finite set of vectors \(S=\{\vec v_1,\dots,\vec v_k\}\) is the set of all linear combinations of \(S\).
For any finite set \(S\subseteq V\), \(\text{Span }S\le V\).
In fact, \(\text{Span }S\) is the smallest subspace of V containing every vector of \(S\).

May 7 Lecture. Links to the board and the recording.

Linear Independence¶

Definition:
A (finite, non-empty) set \(S\subseteq V\) is linearly independent if no vector in \(S\) is a Linear Combination of other vectors in \(S\).
Elaborated Definition:
A (finite, non-empty) set \(S\subseteq V\) is linearly independent if the following is true:
If \(c_1\vec v_1+\dots +c_k\vec v_k=\vec 0\) then \(c_1=\dots=c_k=0\).
Example:
\(S=\{(1, 2, 3,), (0,-1,4)\}\)
Suppose: \(c_1\cdot(1,2,3)+c_2\cdot(0,-1,4)=(0,0,0)\)
That means that:

Solving for \(c_1, c_2, c_3\), we will see that they all equal zero.
This is proof of linear independence.
Example:
\(S=\{(1, 2, 3), (-4,-8,-12)\}\)
Suppose: \(c_1\cdot(1,2,3)+c_2\cdot(-4,-8,-12)=(0,0,0)\)
Extracting equations for c_1, c_2, c_3 will yield the following:
\(\begin{aligned}&c_1-4c_2&=0\\2&c_1-8c_2&=0\\3&c_1-12c_2&=0\end{aligned}\)
The second and third equations are both multiples of the first equation, so we can ignore them. we are left with \(c_1=4c_2\), which has an infinite number of solutions. This means that \(S\) is linearly dependent.
In other words, \(S\) is linearly independent if the only linear combination of \(S\) equal to \(\vec 0\) is the trivial linear combination (i.e., all scalars are 0).
By this definition, \(S=\vec 0\) is not independent, because if \(c\cdot(0, 0, 0)=(0, 0, 0)\), then \(c\) need not be \(0\), can be any scalar.
Any other set of one vector is independent.

How to check if a set of vectors is Linearly Independent¶

\(S_1=\{(1,3,5),(2,4,6),(-1,1,3)\}\)
To check if \(S_1\) is independent:
Suppose \(c_1\begin{pmatrix}1\\3\\5\end{pmatrix}+c_2\begin{pmatrix}2\\4\\6\end{pmatrix}+c_3\begin{pmatrix}-1\\1\\3\end{pmatrix}=\begin{pmatrix}0\\0\\0\end{pmatrix}\) (We wish to show: \(c_1=c_2=c_3=0\))
Extract equations for \(c_1, c_2, c_3\):
\(\begin{aligned}&c_1+2c_2-c_3&=0\\3&c_1-4c_2+c_3&=0\\5&c_1+6c_2+3c_3&=0\end{aligned}\)
Solve system using coefficient matrix:
\(\begin{bmatrix}1&2&-1\\3&4&1\\5&6&3\end{bmatrix}\)
There are two possibilities for a homogeneous system:

• There is a unique solution \((0,0,0)\), which means \(c_1=c_2=c_3=0\), which proves \(S\) to be independent.
• There are infinitely many solutions, which means \(c_1=c_2=c_3=0\) is not true, which proves \(S\) to be dependent.
  Solving the matrix as such…

Basis and Dimension¶

Definition:
A basis of a vector space \(V\) is a linearly independent set \(S\) which spans \(V\). (i.e. Span(\(S\)) \(=V\))
Examples:
In \(\R^2\), \(S_1=\{(1,0),(0,1)\}\)
This set is independent (easy to check)
This set spans \(\R^2\): \(\forall (a,b)\in\R^2:(a,b)=a(1,0)+b(0,1)\)
Therefore, \(S_1\) is a basis of \(\R^2\)
In \(\R^2\), \(S_1=\{(1,1),(-3,5)\}\)
Independent? Neither is a multiple of the other, so yes.
Spanning? \(\forall (a,b)\in\R^2\stackrel?\exists c_1,c_2\in\R\mid(a,b)=c_1(1,1)+c_2(-3,5))\)
Yes (We’ll come back to this later.)
Therefore, \(S_2\) is a basis of \(\R^2\)
In \(\R^2\), \(S_3=\{(1,0),(0,1), (2,3)\}\rightarrow\) spans, but isn’t independent
In \(\R^2\), \(S_4=\{(1,3)\}\rightarrow\) independent, but doesn’t span
In \(\R^2\), \(S_5=\{(1,-1),(-3,3)\}\rightarrow\) doesn’t span and isn’t independent
Claims:
Proofs omitted. See pp. 12, 13

1. If \(B=\{\vec v_1,\dots\vec v_n\}\) is a basis of \(V\), then any \(\vec v\in V\) has a unique representation as a linear combination of \(B\)
2. If \(S\subseteq\R^n, S=\{\vec v_1,\dots\vec v_n\}\) (i.e., \(S\) is a set of \(n\) vectors in \(\R^n\)) then \(S\) spans \(\R^n\) iff \(S\) is independent.
   • Hence: it is enough to check one condition (independent or spanning) to determine if \(S\) is a basis

May 11 Tirgul. Links to the board and the recording.