Dimensions and metrics for evaluating recommendation systems

Iman Avazpour, Teerat Pitakrat, Lars Grunske, John Grundy

Research output: Chapter in Book/Report/Conference proceedingChapter (Book)Researchpeer-review

74 Citations (Scopus)


Recommendation systems support users and developers of various computer and software systems to overcome information overload, perform information discovery tasks, and approximate computation, among others. They have recently become popular and have attracted a wide variety of application scenarios ranging from business process modeling to source code manipulation. Due to this wide variety of application domains, different approaches and metrics have been adopted for their evaluation. In this chapter, we review a range of evaluation metrics and measures as well as some approaches used for evaluating recommendation systems. The metrics presented in this chapter are grouped under sixteen different dimensions, e.g., correctness, novelty, coverage. We review these metrics according to the dimensions to which they correspond. A brief overview of approaches to comprehensive evaluation using collections of recommendation system dimensions and associated metrics is presented. We also provide suggestions for key future research and practice directions.

Original languageEnglish
Title of host publicationRecommendation Systems in Software Engineering
EditorsMartin P. Robillard, Walid Maalej, Robert J. Walker, Thomas Zimmermann
Place of PublicationBerlin Germany
Number of pages29
ISBN (Electronic)9783642451355
ISBN (Print)9783642451348
Publication statusPublished - 2014
Externally publishedYes

Cite this