# Advanced Calculus

Jump to navigation
Jump to search

Advanced Calculus as taught in Fitzpatrick's book.

## Contents

## Sequences

## Topology

### Closed

The folllowing definitions of Closed Sets are equivalent.

- (Order)
- (Sequences) A set \(\displaystyle S\) is clsoed if it contains all its limit points. That is \(\displaystyle \forall \{x_i\} \subseteq S\), \(\displaystyle \{x_i\} \rightarrow x_0 \implies x_0 \in S\).
- (Topology)

- Notes

- Union of infinitely many closed sets can be open.
- Intersection of infinitely many open sets can be closed.
- \(\displaystyle \{\}\) and \(\displaystyle \mathbb{R}\) are both open and closed

### Compact

Compactness is a generalization of closed and bounded.

- Definitions

- (Sequence) A set is sequentially compact if for every sequence from the set, there exists a subsequence which converges to a point in the set.
- (Topology) A set is compact if for every covering by infinitely many open sets, there exists a covering by a finite subset of the open sets.

- Notes

- A set is sequentially compact iff it is closed and bounded

### Metric Space

## Continuity

### Definitions of Continuity

The following definitions of Continuity are equivalent.

- (Order) A function \(\displaystyle f\) is continuous at \(\displaystyle x_0\) if for all \(\displaystyle \epsilon\) there exists \(\displaystyle \delta\) such that \(\displaystyle |x - x_0| \leq \delta \implies |f(x) - f(x_0)| \leq \epsilon\)
- (Sequences) A function \(\displaystyle f\) is continuous at \(\displaystyle x_0\) if \(\displaystyle \{x_n\} \rightarrow x_0 \implies \{f(x_n)\} \rightarrow f(x_0)\)
- (Topology) A function \(\displaystyle f\) is continuous at \(\displaystyle x_0\) if for all open sets \(\displaystyle V\) s.t. \(\displaystyle f(x_0) \in V\), \(\displaystyle f^{-1}(V)\) is an open set.
- The preimage of an open set is open.
- Continuous functions map compact sets to compact sets.

## Differentiation

## Integration

## Approximation

## Series

## Inverse Function Theorem

## Implicit Function Theorem

## Line and Surface Integrals

## Derivatives with respect to vectors and matrices

Not typically covered in undergraduate analysis and calculus classes but necessary for machine learning.

See The Matrix Cookbook

- \(\displaystyle \partial_{x} x^t x = \partial_{x} \operatorname{Tr}(x^t x) = \operatorname{Tr}( (\partial x)^t x + x^t (\partial x)) = 2 * \operatorname{Tr}(x^t (\partial x)) = 2x\)