Metrics¶
Elliot provides 36 evaluation metrics, partitioned into seven families: Accuracy, Rating-Error, Coverage, Novelty, Diversity, Bias, and Fairness. It is worth mentioning that Elliot is the framework that exposes both the largest number of metrics and the only one considering bias and fairness measures. Moreover, the user can choose any metric to drive the model selection and the tuning.
All the metrics inherit from a common abstract class:
|
This class represents the implementation of the Precision recommendation metric. |
Accuracy
|
Area Under the Curve |
|
Group Area Under the Curve |
|
Limited Area Under the Curve |
|
Sørensen–Dice coefficient |
|
F-Measure |
Extended F-Measure |
|
Hit Rate |
|
|
Mean Average Precision |
|
Mean Average Recall |
|
Mean Reciprocal Rank |
|
normalized Discounted Cumulative Gain |
normalized Discounted Cumulative Gain |
Bias
|
Average coverage of long tail items |
|
Average percentage of long tail items |
|
Average Recommendation Popularity |
|
Popularity-based Ranking-based Equal Opportunity |
Extended Popularity-based Ranking-based Equal Opportunity |
|
|
Popularity-based Ranking-based Statistical Parity |
Extended Popularity-based Ranking-based Statistical Parity |
Coverage
Item Coverage |
|
Number of Recommendations Retrieved |
|
User Coverage |
|
|
User Coverage on Top-N rec. |
Diversity
Gini Index |
|
Shannon Entropy |
|
Subtopic Recall |
Fairness
Bias Disparity - Standard |
|
Bias Disparity - Bias Recommendations |
|
Bias Disparity - Bias Source |
|
Item MAD Ranking-based |
|
Item MAD Rating-based |
|
User MAD Ranking-based |
|
User MAD Rating-based |
|
|
Ranking-based Equal Opportunity |
|
Ranking-based Statistical Parity |
Novelty
|
Expected Free Discovery (EFD) |
Extended EFD |
|
|
Expected Popularity Complement (EPC) |
Extended EPC |
Rating
|
Mean Absolute Error |
|
Mean Squared Error |
|
Root Mean Squared Error |