elliot.evaluation.metrics.bias.pop_rsp package

Submodules

elliot.evaluation.metrics.bias.pop_rsp.extended_pop_rsp module

This is the implementation of the Popularity-based Ranking-based Statistical Parity (RSP) metric. It proceeds from a user-wise computation, and average the values over the users.

class elliot.evaluation.metrics.bias.pop_rsp.extended_pop_rsp.ExtendedPopRSP(recommendations, config, params, eval_objects, additional_data)

Bases: elliot.evaluation.metrics.base_metric.BaseMetric

Extended Popularity-based Ranking-based Statistical Parity

This class represents the implementation of the Extended Popularity-based Ranking-based Statistical Parity (RSP) recommendation metric.

For further details, please refer to the paper

To compute the metric, add it to the config file adopting the following pattern:

complex_metrics:
- metric: ExtendedPopRSP

eval()

Evaluation function :return: the overall averaged value of PopRSP

static name()

Metric Name Getter :return: returns the public name of the metric

elliot.evaluation.metrics.bias.pop_rsp.pop_rsp module

This is the implementation of the Popularity-based Ranking-based Statistical Parity (RSP) metric. It proceeds from a user-wise computation, and average the values over the users.

class elliot.evaluation.metrics.bias.pop_rsp.pop_rsp.PopRSP(recommendations, config, params, eval_objects)

Bases: elliot.evaluation.metrics.base_metric.BaseMetric

Popularity-based Ranking-based Statistical Parity

This class represents the implementation of the Popularity-based Ranking-based Statistical Parity (RSP) recommendation metric.

For further details, please refer to the paper

\[\mathrm {RSP}=\frac{{std}\left(P\left(R @ k \mid g=g_{1}\right), \ldots, P\left(R @ k \mid g=g_{A}\right)\right)} {{mean}\left(P\left(R @ k \mid g=g_{1}\right), \ldots, P\left(R @ k \mid g=g_{A}\right)\right)}\]

:math P(R @ k mid g=g_{A})) = frac{sum_{u=1}^{N} sum_{i=1}^{k} G_{g_{a}}(R_{u, i})} {sum_{u=1}^{N} sum_{i in I backslash I_{u}^{+}} G_{g_{a}}(i)}

\(\sum_{i=1}^{k} G_{g_{a}}\left(R_{u, i}\right)\) calculates how many un-interacted items from group {g_a} are ranked in top-𝑘 for user u.

\(\sum_{i \in I \backslash I_{u}^{+}} G_{g_{a}}(i)\) calculates how many un-interacted items belong to group {g_a} for u

To compute the metric, add it to the config file adopting the following pattern:

simple_metrics: [PopRSP]

eval()

Evaluation function :return: the overall averaged value of PopRSP

static name()

Metric Name Getter :return: returns the public name of the metric

Module contents