Evaluating Fairness in Information Retrieval Systems: A Study on the Performance of the FAIR Metric

Abstract

In the digital era, the use of information retrieval (IR) technologies has surged, enabling users to access vast amounts of data quickly. However, concerns have arisen regarding bias and unfairness in these systems, leading to unequal treatment and outcomes for different user groups, such as racial, gender, or socioeconomic bias. To ensure equitable access to information, it is crucial to establish a fair IR system.

This thesis focuses on the FAIR metric proposed by Gao et al, 2022 and investigates its performance by replacing standard information retrieval (IR) utility metrics. It explores the impact of different metrics on the overall performance of FAIR, providing a comprehensive analysis of its effectiveness in evaluating fairness in IR systems.