Advanced Calculus

Advanced Calculus as taught in Fitzpatrick's book.

Topology

Closed

The folllowing definitions of Closed Sets are equivalent.

• (Order)
• (Sequences) A set \(\displaystyle S\) is clsoed if it contains all its limit points. That is \(\displaystyle \forall \{x_i\} \subseteq S\), \(\displaystyle \{x_i\} \rightarrow x_0 \implies x_0 \in S\).
• (Topology)

Notes

• Union of infinitely many closed sets can be open.
• Intersection of infinitely many open sets can be closed.
• \(\displaystyle \{\}\) and \(\displaystyle \mathbb{R}\) are both open and closed

Compact

Compactness is a generalization of closed and bounded.

Definitions

• (Sequence) A set is sequentially compact if for every sequence from the set, there exists a subsequence which converges to a point in the set.
• (Topology) A set is compact if for every covering by infinitely many open sets, there exists a covering by a finite subset of the open sets.

Notes

• A set is sequentially compact iff it is closed and bounded

Continuity

Definitions of Continuity

The following definitions of Continuity are equivalent.

• (Order) A function \(\displaystyle f\) is continuous at \(\displaystyle x_0\) if for all \(\displaystyle \epsilon\) there exists \(\displaystyle \delta\) such that \(\displaystyle |x - x_0| \leq \delta \implies |f(x) - f(x_0)| \leq \epsilon\)
• (Sequences) A function \(\displaystyle f\) is continuous at \(\displaystyle x_0\) if \(\displaystyle \{x_n\} \rightarrow x_0 \implies \{f(x_n)\} \rightarrow f(x_0)\)
• (Topology) A function \(\displaystyle f\) is continuous at \(\displaystyle x_0\) if for all open sets \(\displaystyle V\) s.t. \(\displaystyle f(x_0) \in V\), \(\displaystyle f^{-1}(V)\) is an open set.
  • The preimage of an open set is open.
  • Continuous functions map compact sets to compact sets.

Derivatives with respect to vectors and matrices

Not typically covered in undergraduate analysis and calculus classes but necessary for machine learning.
See The Matrix Cookbook

• \(\displaystyle \partial_{x} x^t x = \partial_{x} \operatorname{Tr}(x^t x) = \operatorname{Tr}( (\partial x)^t x + x^t (\partial x)) = 2 * \operatorname{Tr}(x^t (\partial x)) = 2x\)