Advanced Calculus as taught in Fitzpatrick's book.

## Topology

### Closed

The folllowing definitions of Closed Sets are equivalent.

• (Order)
• (Sequences) A set $S$ is clsoed if it contains all its limit points. That is $\forall \{x_{i}\}\subseteq S$ , $\{x_{i}\}\rightarrow x_{0}\implies x_{0}\in S$ .
• (Topology)
Notes
• Union of infinitely many closed sets can be open.
• Intersection of infinitely many open sets can be closed.
• $\{\}$ and $\mathbb {R}$ are both open and closed

### Compact

Compactness is a generalization of closed and bounded.

Definitions
• (Sequence) A set is sequentially compact if for every sequence from the set, there exists a subsequence which converges to a point in the set.
• (Topology) A set is compact if for every covering by infinitely many open sets, there exists a covering by a finite subset of the open sets.
Notes
• A set is sequentially compact iff it is closed and bounded

## Continuity

### Definitions of Continuity

The following definitions of Continuity are equivalent.

• (Order) A function $f$ is continuous at $x_{0}$ if for all $\epsilon$ there exists $\delta$ such that $|x-x_{0}|\leq \delta \implies |f(x)-f(x_{0})|\leq \epsilon$ • (Sequences) A function $f$ is continuous at $x_{0}$ if $\{x_{n}\}\rightarrow x_{0}\implies \{f(x_{n})\}\rightarrow f(x_{0})$ • (Topology) A function $f$ is continuous at $x_{0}$ if for all open sets $V$ s.t. $f(x_{0})\in V$ , $f^{-1}(V)$ is an open set.
• The preimage of an open set is open.
• Continuous functions map compact sets to compact sets.

## Derivatives with respect to vectors and matrices

Not typically covered in undergraduate analysis and calculus classes but necessary for machine learning.
See The Matrix Cookbook

• $\partial _{x}x^{t}x=\partial _{x}\operatorname {Tr} (x^{t}x)=\operatorname {Tr} ((\partial x)^{t}x+x^{t}(\partial x))=2*\operatorname {Tr} (x^{t}(\partial x))=2x$ 