Neural Network Compression: Difference between revisions
m David moved page Private:Neural Network Compression to Neural Network Compression over redirect |
|||
| Line 5: | Line 5: | ||
The idea here is to measure how sensitive each neuron is. | The idea here is to measure how sensitive each neuron is. | ||
I.e., if you remove the neuron, how will it change the output? | I.e., if you remove the neuron, how will it change the output? | ||
* Mozer and Smolensky (1988)<ref name="mozer1988skeletonization"></ref> use a gate for each neuron. Then the sensitivity and be estimated with the derivative w.r.t the gate. | |||
==Factorization== | ==Factorization== | ||
Revision as of 20:39, 2 February 2021
Brief survey on neural network compression techniques.
Pruning
Sensitivity Methods
The idea here is to measure how sensitive each neuron is.
I.e., if you remove the neuron, how will it change the output?
- Mozer and Smolensky (1988)[1] use a gate for each neuron. Then the sensitivity and be estimated with the derivative w.r.t the gate.
Factorization
Resources
Surveys
- Pruning algorithms a survey (1993) by Russel Reed
- A Survey of Model Compression and Acceleration for Deep Neural Networks (2017) by Cheng et al.
- ↑ Cite error: Invalid
<ref>tag; no text was provided for refs namedmozer1988skeletonization