2) Try to present a mix of familiar and unknown items to the user.
E.g. Given a user that really likes The Rolling Stones, recommend to her:
The Beatles
The Who
Mick Jagger
Led Zeppelin
etc.
Most of these recommendations are obvious, so why do I need a recommender, then?
However, it would be great to get some more “interesting” recommendations, such as:
Robert Johnson
Muddy Waters
Chuck Berry
The Faces
The Georgia Satellites
Sol Lagarto
Diamond Dogs
The Dogs d’Amour
…
I’m particularly interested in the second approach, as it really helps users discover unknown items that otherwise would have been very difficult to get into.
Some other references for diversity in recommendation:
Measuring playlist diversity for recommendation systems by Malcolm Slaney and William White
Oscar’s (http://mtg.upf.edu/~ocelma) thesis (when he finally publishes it
Beyond Algorithms: An HCI Perspective on Recommender Systems
Kirsten Swearingen & Rashmi Sinha
Evaluating Collaborative Filtering
Recommender Systems – Herlocker, Konstan, Terveen, Riedl
Isn’t lack of diversity some sort of overfitting?
If not how you would you define the difference?
Very interesting topic Daniel!
I see two approaches regarding ‘diversity’ in RS:
1) Try to avoid too similar items in the top-N recommended items.
E.g Avoid this: given a user that loves Woody Allen, then recommend Woody Allen films that are not in her profile.
Also know as “The White Album effect”:
http://www.amazon.com/Beatles-White-Album/dp/B000002UAX
2) Try to present a mix of familiar and unknown items to the user.
E.g. Given a user that really likes The Rolling Stones, recommend to her:
The Beatles
The Who
Mick Jagger
Led Zeppelin
etc.
Most of these recommendations are obvious, so why do I need a recommender, then?
However, it would be great to get some more “interesting” recommendations, such as:
Robert Johnson
Muddy Waters
Chuck Berry
The Faces
The Georgia Satellites
Sol Lagarto
Diamond Dogs
The Dogs d’Amour
…
I’m particularly interested in the second approach, as it really helps users discover unknown items that otherwise would have been very difficult to get into.
Cheers, Oscar
Oscar: Good point. And this stresses the fact that “probabilistically less accurate recommendations” may prove far more valuable.
Some other references for diversity in recommendation:
Measuring playlist diversity for recommendation systems by Malcolm Slaney and William White
Oscar’s (http://mtg.upf.edu/~ocelma) thesis (when he finally publishes it
Beyond Algorithms: An HCI Perspective on Recommender Systems
Kirsten Swearingen & Rashmi Sinha
Evaluating Collaborative Filtering
Recommender Systems – Herlocker, Konstan, Terveen, Riedl
Here are some papers which I also consider relevant:
* Zhai, Cohen, Lafferty: “Beyond Independent Relevance: Methods and Evaluation Metrics for Subtopic Retrieval” SIGIR 03
* Song, Tian, Huang: “Improving the Image Retrieval Result Via Topic Coverage Grap” PCM2006
* Chen, Karger: “Less is More: Probabilistic Models for Retrieving Fewer Relevant Documents” SIGIR06
* Tang, Arni, Sanderson, Clough: “Building a Diversity Featured Search System by Fusing Existing Tools”
* Arni, Clough, Sanderson, Grubinger: “Overview of the ImageCLEF 2008 Photographic Retrieval Task”
* Clarke, Kolla, Cormack, Vechtomova, Ashkan, Büttcher, MacKinnon: “Novelty and Diversity in Information Retrieval Evaluation” SIGIR08
* Xu, Yin: “Novelty and Topicality in Interactive Information Retrieval”
You might also be interested in the photo retrieval task of ImageCLEF 2008, where diversity was taken into account.
Wow, Beyond Algorithms is great!
Fig 7 is particularly interesting in the debate about whether CF will narrow our interests.
It also shows the importance of transparency (Table 1 + Fig 10).
I would add this to your list of papers Daniel:
Recommender systems and their impact on sales diversity
http://portal.acm.org/citation.cfm?id=1250910.1250939
http://doi.acm.org/10.1145/1250910.1250939