Reweighing for Fairness

Reweighing is a fairness technique that assigns different importance weights to training examples based on their group membership and outcome. If women with approved loans are underrepresented in your data, you give those examples higher weight so the model pays more attention to them. If men with approved loans are overrepresented, you give those examples lower weight. This rebalancing happens before training, so the resulting model learns from a "fairer" version of the data without changing the actual records. It works like giving certain students' homework answers more weight when grading to compensate for past disadvantages. Notice this: reweighing adjusts the data's influence, not the data itself.

Reweighing works by calculating weights for each combination of protected attribute (like gender or race) and outcome label (approved/denied, hired/not hired). First, you compute the expected probability of each combination if the data were completely unbiased. Then you assign weights equal to the ratio of expected to observed frequencies—examples that appear less often than they should get higher weights, while overrepresented examples get lower weights.

During model training, these weights tell the algorithm how much to consider each example. If qualified women are underrepresented in your training data compared to what would be expected in a fair distribution, their examples receive proportionally higher weights during training, while overrepresented groups receive lower weights. The model trains on the same data but learns different decision patterns because it's now "listening" more carefully to previously underrepresented groups.

This approach applies across many domains. In hiring, you might reweigh resumes from underrepresented groups who succeeded in past roles to counteract historical exclusion. In healthcare, you could reweigh patient outcomes by demographic groups to ensure treatment recommendations aren't skewed by unequal access to care. In lending, reweighing can address legacy discrimination where certain groups received fewer loans regardless of creditworthiness. The key is that reweighing operates at the data level, making it compatible with almost any machine learning algorithm.

Reweighing was formalized as a bias mitigation technique by Kamiran and Calders (2012), who studied data preprocessing techniques including reweighing for discrimination-aware classification, examining the theoretical trade-offs between accuracy and non-discrimination [1]. The method operates by computing weights that equalize the joint distribution of protected attributes and labels between groups, effectively creating a balanced "virtual" training dataset.

• Kamiran and Calders (2012): Data preprocessing techniques can reduce discriminatory outcomes in classification tasks while preserving accuracy, compared to unmitigated baselines [1].
• Bellamy et al. (2018): Reweighing is implemented as a core preprocessing algorithm in AI Fairness 360, demonstrating effectiveness across credit scoring, hiring, and criminal justice datasets with minimal computational overhead [2].
• Mehrabi et al. (2021): Reweighing is classified as a preprocessing method that addresses bias at the data level, making it compatible with any downstream classifier but limited to addressing only representation disparities rather than feature-level biases [3].

Reweighing has important constraints. It only addresses disparate impact arising from unequal representation in training data, not from biased features or labels themselves. If the outcome labels are contaminated by human discrimination (such as biased performance reviews), reweighing will redistribute that unfairness rather than eliminate it. The method assumes sensitive attributes are available and accurately recorded—raising privacy concerns in some applications. Critically, reweighing optimizes for demographic parity, which may conflict with other fairness definitions like equalized odds or calibration; there's also a fairness-accuracy tradeoff: reweighing involves trade-offs that may affect model performance, and the weights may become unstable for rare group-outcome combinations. Finally, reweighing is a technical fix that doesn't address root causes of discrimination in data collection or system design.