[KDD 2020] On Sampling Top-K Recommendation Evaluation - CrossMinds.ai
[KDD 2020] On Sampling Top-K Recommendation Evaluation
Aug 13, 202010 views
Dong Li
Recently, Rendle has warned that the use of sampling-based topπ‘˜,metrics might not suffice. This throws a number of recent studies,on deep learning-based recommendation algorithms, and classic,non-deep-learning algorithms using such a metric, into jeopardy.,In this work, we thoroughly investigate the relationship between,the sampling and global top𝐾,Hit-Ratio (HR, or Recall), originally,proposed by Koren [,2,] and extensively used by others. By formulating the problem of aligning sampling topπ‘˜,(,𝑆𝐻𝑅,@,π‘˜,) and global,top𝐾,(,𝐻𝑅,@,𝐾,) Hit-Ratios through a mapping function,𝑓,, so that,𝑆𝐻𝑅,@,π‘˜,β‰ˆ,𝐻𝑅,@,𝑓,(,π‘˜,),, we demonstrate both theoretically and experimentally that the sampling topπ‘˜,Hit-Ratio provides an accurate,approximation of its global (exact) counterpart, and can consistently predict the correct winners (the same as indicate by their,corresponding global Hit-Ratios).