5,332
edits
Deep Learning: Difference between revisions
→Theory of self-supervised learning
| Line 1,817: | Line 1,817: | ||
# Case 1: H is a discrete variable then <math>g^*(x, z) = \eta^*(x)^t \psi(z^*)</math> | # Case 1: H is a discrete variable then <math>g^*(x, z) = \eta^*(x)^t \psi(z^*)</math> | ||
# Case 2: There exists <math>\eta,\psi</math> such that <math>E(\eta(x)^t \psi(z) - g^*(x,z))^2 \leq o(\frac{1}{m})</math>. | # Case 2: There exists <math>\eta,\psi</math> such that <math>E(\eta(x)^t \psi(z) - g^*(x,z))^2 \leq o(\frac{1}{m})</math>. | ||
==Meta Learning== | |||
In many applications, we don't have a large training dataset. However, humans can adapt and learn ''on the fly''. | |||
The key is to use prior knowledge in performing new tasks. | |||
How can we train AI/ML models similarly? | |||
Goal of meta learning: Train a model on different learning tasks such that it can solve new tasks using only a small number of training samples. | |||
Few shot classification problem: | |||
Inputs: <math>D_{metatrain} = \{(D_i^{train}, D_i^{test})\}_{i=1}^{n}</math>. | |||
<math> | |||
\begin{cases} | |||
D_{i}^{train} = \{(x_j^i, y_j^i) \}_{j=1}^{k}\\ | |||
D_{i}^{test} = \{(x_j^i, y_j^i) \}_{j=1}^{k'} | |||
\end{cases} | |||
</math> | |||
==Misc== | ==Misc== | ||