Ranking: Difference between revisions
| (3 intermediate revisions by the same user not shown) | |||
| Line 5: | Line 5: | ||
===Point-wise ranking=== | ===Point-wise ranking=== | ||
In point-wise ranking, you have some scores for you document <math>y_i</math> so you can train your model <math>f</math> to predict such scores in a | In point-wise ranking, you have some scores for you document <math>y_i</math> so you can train your model <math>f</math> to predict such scores in a supervised manner. | ||
===Pair-wise ranking=== | |||
If you data is of the form: <math>y(x_a) > y(x_b)</math> then you can train so that your model maximizes <math>f(x_a) - f(x_b)</math> using a hinge loss: | |||
<math> | |||
\begin{equation} | |||
L(x_a, x_b) = max(0, 1-(f(x_a) - f(x_b))) | |||
\end{equation} | |||
</math> | |||
===Listwise ranking=== | |||
Use something like [https://auai.org/uai2014/proceedings/individuals/164.pdf ListMLE] | |||
==Metrics== | ==Metrics== | ||
| Line 36: | Line 46: | ||
{{main | Wikipedia: Mean reciprocal rank}} | {{main | Wikipedia: Mean reciprocal rank}} | ||
If you only have one correct answer which is placed in rank <math>i</math> then the reciprocal rank is <math>1/i</math>.<br> | If you only have one correct answer which is placed in rank <math>i</math> then the reciprocal rank is <math>1/i</math>.<br> | ||
For multiple queries and results, the mean reciprocal rank is simply <math>\mean(1/rank)</math>. | For multiple queries and results, the mean reciprocal rank is simply <math>\operatorname{mean}(1/rank)</math>. | ||