Vector Space

\( \newcommand{\states}{\mathcal{S}} \newcommand{\actions}{\mathcal{A}} \newcommand{\observations}{\mathcal{O}} \newcommand{\rewards}{\mathcal{R}} \newcommand{\traces}{\mathbf{e}} \newcommand{\transition}{P} \newcommand{\reals}{\mathbb{R}} \newcommand{\naturals}{\mathbb{N}} \newcommand{\complexs}{\mathbb{C}} \newcommand{\field}{\mathbb{F}} \newcommand{\numfield}{\mathbb{F}} \newcommand{\expected}{\mathbb{E}} \newcommand{\var}{\mathbb{V}} \newcommand{\by}{\times} \newcommand{\partialderiv}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\defineq}{\stackrel{{\tiny\mbox{def}}}{=}} \newcommand{\defeq}{\stackrel{{\tiny\mbox{def}}}{=}} \newcommand{\eye}{\Imat} \newcommand{\hadamard}{\odot} \newcommand{\trans}{\top} \newcommand{\inv}{{-1}} \newcommand{\argmax}{\operatorname{argmax}} \newcommand{\Prob}{\mathbb{P}} \newcommand{\avec}{\mathbf{a}} \newcommand{\bvec}{\mathbf{b}} \newcommand{\cvec}{\mathbf{c}} \newcommand{\dvec}{\mathbf{d}} \newcommand{\evec}{\mathbf{e}} \newcommand{\gvec}{\mathbf{g}} \newcommand{\hvec}{\mathbf{h}} \newcommand{\lvec}{\mathbf{l}} \newcommand{\mvec}{\mathbf{m}} \newcommand{\nvec}{\mathbf{n}} \newcommand{\pvec}{\mathbf{p}} \newcommand{\qvec}{\mathbf{q}} \newcommand{\rvec}{\mathbf{r}} \newcommand{\svec}{\mathbf{s}} \newcommand{\uvec}{\mathbf{u}} \newcommand{\vvec}{\mathbf{v}} \newcommand{\wvec}{\mathbf{w}} \newcommand{\xvec}{\mathbf{x}} \newcommand{\yvec}{\mathbf{y}} \newcommand{\zvec}{\mathbf{z}} \newcommand{\Amat}{\mathbf{A}} \newcommand{\Bmat}{\mathbf{B}} \newcommand{\Cmat}{\mathbf{C}} \newcommand{\Dmat}{\mathbf{D}} \newcommand{\Emat}{\mathbf{E}} \newcommand{\Fmat}{\mathbf{F}} \newcommand{\Imat}{\mathbf{I}} \newcommand{\Pmat}{\mathbf{P}} \newcommand{\Umat}{\mathbf{U}} \newcommand{\Vmat}{\mathbf{V}} \newcommand{\Wmat}{\mathbf{W}} \newcommand{\Xmat}{\mathbf{X}} \newcommand{\Qmat}{\mathbf{Q}} \newcommand{\thetavec}{\boldsymbol{\theta}} \newcommand{\phivec}{\boldsymbol{\phi}} \newcommand{\muvec}{\boldsymbol{\mu}} \newcommand{\sigmavec}{\boldsymbol{\sigma}} \newcommand{\jacobian}{\mathbf{J}} \newcommand{\ind}{\perp!!!!\perp} \)

tags

Linear Algebra, Math

A vector space is a set of vectors together with rules for vector addition and multiplication by scalars. The addition and multiplication rules must be consistent with the following properties (for \(x\in\mathbb{F}^n\), \(c\in\mathbb{F}\), \(\mathbb{F} = \{\mathbb{R}, \mathbb{C}\}\) ):

1. \(\xvec+\yvec = \yvec+\xvec\)
2. \(\xvec+(\yvec+\zvec) = (\xvec+\yvec) + \zvec\)
3. There is a unique zero vector \(\mathbf{0} + \xvec = \xvec\)
4. For each \(\xvec\) there is a unique vector \(-\xvec\) such that \(\xvec + (-\xvec) = \mathbf{0}\).
5. \(1\xvec = \xvec\)
6. \((c_1c_2)\xvec = c_1(c_2\xvec)\)
7. \(c(\xvec + \yvec) = c\xvec + c\yvec\)
8. \((c_1 + c_2)\xvec = c_1\xvec + c_2\xvec\)

Vector Subspaces

A vector subspace is a non-empyt subset of a vector space which is closed under the same linear combination rules. The most trivial vector subspace of \(\reals^n\) is the set containing the zero vector.

Column (row) space of a matrix

The column space of a matrix is all possible linear combinations of the columns of a matrix. The row space is the same except with linear combinations of the rows of a matrix.

Nullspace of a matrix

The nullspace is:

• A subspace of \(\reals^M\)

Dual Space

The dual vector space is the set \(V^\star\) of all linear functionals \(f: V \mapsto \mathbb{F}\). Since linear functionals are vector space homomorphisms this is sometimes denoted as \(Hom(V, \mathbb{F})\). The dual space is itself a vector space, meaning (for \(\phi, \psi \in V^\star\) and \(\xvec \in V\))

1. \((\phi + \psi)(\xvec) = \phi(\xvec) + \psi(\xvec)\)
2. \((\alpha \phi)(\xvec) = \alpha\phi(\xvec)\).

The dual vector space has the same dimensions as the original vector space, and has a canonical dual basis (or cobasis) \(\theta^j\) defined by the original basis \(e_i\)

\[\langle e_i, \theta^j \rangle = \delta^j_i \quad \triangleright \delta^j_i \text{ is the Kronecker delta function}\].

Notation gets a bit confusing in higher order linear algebra and tensor algebra. Typically, I will attempt to denote an index of the cobasis as an upstairs index while the basis as a downstairs index.