# Statistics

Statistics

## Contents

## Estimation

### Method of Moments Estimator

Sometimes referred to as MME or MMO

- Calculate your population moments in terms of your parameters
- Then invert to get your parameters as a function of your moments
- Replace population moments with sample moments

### Maximum Likelihood Estimator

(MLE) Maximum Likelihood Estimator

- Write out the likelihood function
- (Optional) Write out the log-likelihood function
- Take the derivative of the log-likelihood function w.r.t
- Find the maximum of the log-likelihood function by setting the first derivative to 0
- (Optional) Make sure it is the maximum by checking that the Hessian is positive definite
- Your MLE is the value which maximizes
- Note if the derivative is always 0, then any value is the MLE. If it is always positive, then take the largest possible value.

- Notes

- If is the MLE for then the MLE for is

### Uniformly Minimum Variance Unbiased Estimator (UMVUE)

UMVUE, sometimes called MVUE or UMVU.

See Wikipedia: Lehmann–Scheffé theorem

An unbiased estimator of a complete-sufficient statistics is a UMVUE.

In general, you should find a complete sufficient statistic using the property of exponential families.

Then make it unbiased with some factors to get the UMVUE.

### Properties

#### Unbiased

An estimator is unbiased for if

- is unbiased for but is not consistent

#### Consistent

An estimator is consistent for if it converges in probability to

- Example: is a consistent estimator

- for for but is not unbiased.

### Efficiency

#### Fisher Information

- or if is twice differentiable
- is the fisher information of the sample. Replace with your full likelihood.

#### Cramer-Rao Lower Bound

Given an estimator , let . Then

- Notes

- If is unbiased then

- Our lower bound will be

The efficiency of an unbiased estimator is defined as

### Sufficient Statistics

#### Auxiliary Statistics

## Tests

### Basic Tests

#### T-test

Used to test the mean.

#### F-test

Use to test the ratio of variances.

### Likelihood Ratio Test

See Wikipedia: Likelihood Ratio Test

### Uniformly Most Powerful Test

UMP Test

See Wikipedia: Neyman-Pearson Lemma

### Anova

## Confidence Sets

Confidence Intervals

### Relationship with Tests

## Regression

## Quadratic Forms

## Bootstrapping

Wikipedia

Boostrapping is used to sample from your sample to get a measure of accuracy of your statistics.

### Nonparametric Bootstrapping

In nonparametric bootstrapping, you resample from your sample with replacement.

In this scenario, you don't need to know the family of distributions that your sample comes from.

### Parametric Bootstrapping

In parametric bootstrapping, you learn the distribution parameters of your sample, e.g. with MLE.

Then you can generate samples from that distribution on a computer.