$$\newcommand{\P}[]{\unicode{xB6}} \newcommand{\AA}[]{\unicode{x212B}} \newcommand{\empty}[]{\emptyset} \newcommand{\O}[]{\emptyset} \newcommand{\Alpha}[]{Α} \newcommand{\Beta}[]{Β} \newcommand{\Epsilon}[]{Ε} \newcommand{\Iota}[]{Ι} \newcommand{\Kappa}[]{Κ} \newcommand{\Rho}[]{Ρ} \newcommand{\Tau}[]{Τ} \newcommand{\Zeta}[]{Ζ} \newcommand{\Mu}[]{\unicode{x039C}} \newcommand{\Chi}[]{Χ} \newcommand{\Eta}[]{\unicode{x0397}} \newcommand{\Nu}[]{\unicode{x039D}} \newcommand{\Omicron}[]{\unicode{x039F}} \DeclareMathOperator{\sgn}{sgn} \def\oiint{\mathop{\vcenter{\mathchoice{\huge\unicode{x222F}\,}{\unicode{x222F}}{\unicode{x222F}}{\unicode{x222F}}}\,}\nolimits} \def\oiiint{\mathop{\vcenter{\mathchoice{\huge\unicode{x2230}\,}{\unicode{x2230}}{\unicode{x2230}}{\unicode{x2230}}}\,}\nolimits}$$

Advanced Calculus as taught in Fitzpatrick's book. This is content covered in MATH410 and MATH411 at UMD.

## Topology

### Closed

The folllowing definitions of Closed Sets are equivalent.

• (Order)
• (Sequences) A set $$\displaystyle S$$ is clsoed if it contains all its limit points. That is $$\displaystyle \forall \{x_i\} \subseteq S$$, $$\displaystyle \{x_i\} \rightarrow x_0 \implies x_0 \in S$$.
• (Topology)
Notes
• Union of infinitely many closed sets can be open.
• Intersection of infinitely many open sets can be closed.
• $$\displaystyle \{\}$$ and $$\displaystyle \mathbb{R}$$ are both open and closed

### Compact

Compactness is a generalization of closed and bounded.

Definitions
• (Sequence) A set is sequentially compact if for every sequence from the set, there exists a subsequence which converges to a point in the set.
• (Topology) A set is compact if for every covering by infinitely many open sets, there exists a covering by a finite subset of the open sets.
Notes
• A set is sequentially compact iff it is closed and bounded

## Continuity

### Definitions of Continuity

The following definitions of Continuity are equivalent.

• (Order) A function $$\displaystyle f$$ is continuous at $$\displaystyle x_0$$ if for all $$\displaystyle \epsilon$$ there exists $$\displaystyle \delta$$ such that $$\displaystyle |x - x_0| \leq \delta \implies |f(x) - f(x_0)| \leq \epsilon$$
• (Sequences) A function $$\displaystyle f$$ is continuous at $$\displaystyle x_0$$ if $$\displaystyle \{x_n\} \rightarrow x_0 \implies \{f(x_n)\} \rightarrow f(x_0)$$
• (Topology) A function $$\displaystyle f$$ is continuous at $$\displaystyle x_0$$ if for all open sets $$\displaystyle V$$ s.t. $$\displaystyle f(x_0) \in V$$, $$\displaystyle f^{-1}(V)$$ is an open set.
• The preimage of an open set is open.
• Continuous functions map compact sets to compact sets.

## Derivatives with respect to vectors and matrices

Not typically covered in undergraduate analysis and calculus classes but necessary for machine learning.
See The Matrix Cookbook

• $$\displaystyle \partial_{x} x^t x = \partial_{x} \operatorname{Tr}(x^t x) = \operatorname{Tr}( (\partial x)^t x + x^t (\partial x)) = 2 * \operatorname{Tr}(x^t (\partial x)) = 2x$$