Ulster University Logo

A Novel Classification Framework for Evaluating Individual and Aggregate Diversity in Top-N Recommendations

Moody, Jennifer and Glass, David H. (2016) A Novel Classification Framework for Evaluating Individual and Aggregate Diversity in Top-N Recommendations. ACM Transactions on Intelligent Systems and Technology, 7 (3). 42:1-42:21. [Journal article]

[img] Text - Accepted Version
[img] Text - Supplemental Material
Restricted to Repository staff only


DOI: 10.1145/2700491


The primary goal of a recommender system is to generate high quality user-centred recommendations. However, the traditional evaluation methods and metrics were developed before researchers understood all the factors that increase user satisfaction. This study is an introduction to a novel user and item classification framework. It is proposed that this framework should be used during user-centred evaluation of recommender systems and the need for this framework is justified through experiments. User profiles are constructed and matched against other users’ profiles to formulate neighbourhoods and generate top-N recommendations.The recommendations are evaluated to measure the success of the process. In conjunction with the framework, a new diversity metric is presented and explained. The accuracy, coverage, and diversity of top-N recommendations is illustrated and discussed for groups of users. It is found that in contradiction to common assumptions, not all users suffer as expected from the data sparsity problem. In fact, the group of users that receive the most accurate recommendations do not belong to the least sparse area of the dataset.

Item Type:Journal article
Keywords:Recommender systems, recommendation accuracy, recommendation quality, performance evaluation metrics, recommendation diversity, collaborative filtering
Faculties and Schools:Faculty of Computing & Engineering
Faculty of Computing & Engineering > School of Computing and Mathematics
Research Institutes and Groups:Computer Science Research Institute
Computer Science Research Institute > Artificial Intelligence and Applications
ID Code:34482
Deposited By: Dr David Glass
Deposited On:06 Jul 2016 14:29
Last Modified:06 Jul 2016 14:29

Repository Staff Only: item control page