Vector Space

$\newcommand{\states}{\mathcal{S}} \newcommand{\actions}{\mathcal{A}} \newcommand{\observations}{\mathcal{O}} \newcommand{\rewards}{\mathcal{R}} \newcommand{\traces}{\mathbf{e}} \newcommand{\transition}{P} \newcommand{\reals}{\mathbb{R}} \newcommand{\naturals}{\mathbb{N}} \newcommand{\complexs}{\mathbb{C}} \newcommand{\field}{\mathbb{F}} \newcommand{\numfield}{\mathbb{F}} \newcommand{\expected}{\mathbb{E}} \newcommand{\var}{\mathbb{V}} \newcommand{\by}{\times} \newcommand{\partialderiv}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\defineq}{\stackrel{{\tiny\mbox{def}}}{=}} \newcommand{\defeq}{\stackrel{{\tiny\mbox{def}}}{=}} \newcommand{\eye}{\Imat} \newcommand{\hadamard}{\odot} \newcommand{\trans}{\top} \newcommand{\inv}{{-1}} \newcommand{\argmax}{\operatorname{argmax}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\avec}{\mathbf{a}} \newcommand{\bvec}{\mathbf{b}} \newcommand{\cvec}{\mathbf{c}} \newcommand{\dvec}{\mathbf{d}} \newcommand{\evec}{\mathbf{e}} \newcommand{\gvec}{\mathbf{g}} \newcommand{\hvec}{\mathbf{h}} \newcommand{\lvec}{\mathbf{l}} \newcommand{\mvec}{\mathbf{m}} \newcommand{\nvec}{\mathbf{n}} \newcommand{\pvec}{\mathbf{p}} \newcommand{\qvec}{\mathbf{q}} \newcommand{\rvec}{\mathbf{r}} \newcommand{\svec}{\mathbf{s}} \newcommand{\uvec}{\mathbf{u}} \newcommand{\vvec}{\mathbf{v}} \newcommand{\wvec}{\mathbf{w}} \newcommand{\xvec}{\mathbf{x}} \newcommand{\yvec}{\mathbf{y}} \newcommand{\zvec}{\mathbf{z}} \newcommand{\Amat}{\mathbf{A}} \newcommand{\Bmat}{\mathbf{B}} \newcommand{\Cmat}{\mathbf{C}} \newcommand{\Dmat}{\mathbf{D}} \newcommand{\Emat}{\mathbf{E}} \newcommand{\Fmat}{\mathbf{F}} \newcommand{\Imat}{\mathbf{I}} \newcommand{\Pmat}{\mathbf{P}} \newcommand{\Umat}{\mathbf{U}} \newcommand{\Vmat}{\mathbf{V}} \newcommand{\Wmat}{\mathbf{W}} \newcommand{\Xmat}{\mathbf{X}} \newcommand{\Qmat}{\mathbf{Q}} \newcommand{\thetavec}{\boldsymbol{\theta}} \newcommand{\phivec}{\boldsymbol{\phi}} \newcommand{\muvec}{\boldsymbol{\mu}} \newcommand{\sigmavec}{\boldsymbol{\sigma}} \newcommand{\jacobian}{\mathbf{J}} \newcommand{\ind}{\perp!!!!\perp}$

tags

Linear Algebra, Math

A vector space is a set of vectors together with rules for vector addition and multiplication by scalars. The addition and multiplication rules must be consistent with the following properties (for $x\in\mathbb{F}^n$, $c\in\mathbb{F}$, $\mathbb{F} = \{\mathbb{R}, \mathbb{C}\}$ ):

1. $\xvec+\yvec = \yvec+\xvec$
2. $\xvec+(\yvec+\zvec) = (\xvec+\yvec) + \zvec$
3. There is a unique zero vector $\mathbf{0} + \xvec = \xvec$
4. For each $\xvec$ there is a unique vector $-\xvec$ such that $\xvec + (-\xvec) = \mathbf{0}$.
5. $1\xvec = \xvec$
6. $(c_1c_2)\xvec = c_1(c_2\xvec)$
7. $c(\xvec + \yvec) = c\xvec + c\yvec$
8. $(c_1 + c_2)\xvec = c_1\xvec + c_2\xvec$

Vector Subspaces

A vector subspace is a non-empyt subset of a vector space which is closed under the same linear combination rules. The most trivial vector subspace of $\reals^n$ is the set containing the zero vector.

Column (row) space of a matrix

The column space of a matrix is all possible linear combinations of the columns of a matrix. The row space is the same except with linear combinations of the rows of a matrix.

Nullspace of a matrix

The nullspace is:

• A subspace of $\reals^M$

Dual Space

The dual vector space is the set $V^\star$ of all linear functionals $f: V \mapsto \mathbb{F}$. Since linear functionals are vector space homomorphisms this is sometimes denoted as $Hom(V, \mathbb{F})$. The dual space is itself a vector space, meaning (for $\phi, \psi \in V^\star$ and $\xvec \in V$)

1. $(\phi + \psi)(\xvec) = \phi(\xvec) + \psi(\xvec)$
2. $(\alpha \phi)(\xvec) = \alpha\phi(\xvec)$.

The dual vector space has the same dimensions as the original vector space, and has a canonical dual basis (or cobasis) $\theta^j$ defined by the original basis $e_i$

$$\langle e_i, \theta^j \rangle = \delta^j_i \quad \triangleright \delta^j_i \text{ is the Kronecker delta function}$$ .

Notation gets a bit confusing in higher order linear algebra and tensor algebra. Typically, I will attempt to denote an index of the cobasis as an upstairs index while the basis as a downstairs index.