Advanced Calculus as taught in Fitzpatrick's book.

## Topology

### Closed

The folllowing definitions of Closed Sets are equivalent.

• (Order)
• (Sequences) A set ${\displaystyle S}$ is clsoed if it contains all its limit points. That is ${\displaystyle \forall \{x_{i}\}\subseteq S}$, ${\displaystyle \{x_{i}\}\rightarrow x_{0}\implies x_{0}\in S}$.
• (Topology)
Notes
• Union of infinitely many closed sets can be open.
• Intersection of infinitely many open sets can be closed.
• ${\displaystyle \{\}}$ and ${\displaystyle \mathbb {R} }$ are both open and closed

### Compact

Compactness is a generalization of closed and bounded.

Definitions
• (Sequence) A set is sequentially compact if for every sequence from the set, there exists a subsequence which converges to a point in the set.
• (Topology) A set is compact if for every covering by infinitely many open sets, there exists a covering by a finite subset of the open sets.
Notes
• A set is sequentially compact iff it is closed and bounded

## Continuity

### Definitions of Continuity

The following definitions of Continuity are equivalent.

• (Order) A function ${\displaystyle f}$ is continuous at ${\displaystyle x_{0}}$ if for all ${\displaystyle \epsilon }$ there exists ${\displaystyle \delta }$ such that ${\displaystyle |x-x_{0}|\leq \delta \implies |f(x)-f(x_{0})|\leq \epsilon }$
• (Sequences) A function ${\displaystyle f}$ is continuous at ${\displaystyle x_{0}}$ if ${\displaystyle \{x_{n}\}\rightarrow x_{0}\implies \{f(x_{n})\}\rightarrow f(x_{0})}$
• (Topology) A function ${\displaystyle f}$ is continuous at ${\displaystyle x_{0}}$ if for all open sets ${\displaystyle V}$ s.t. ${\displaystyle f(x_{0})\in V}$, ${\displaystyle f^{-1}(V)}$ is an open set.
• The preimage of an open set is open.
• Continuous functions map compact sets to compact sets.

## Derivatives with respect to vectors and matrices

Not typically covered in undergraduate analysis and calculus classes but necessary for machine learning.
See The Matrix Cookbook

• ${\displaystyle \partial _{x}x^{t}x=\partial _{x}\operatorname {Tr} (x^{t}x)=\operatorname {Tr} ((\partial x)^{t}x+x^{t}(\partial x))=2*\operatorname {Tr} (x^{t}(\partial x))=2x}$