Algorithmic Bias Sources

Artificial intelligence (AI) systems can produce unfair outcomes because of the examples they learn from, their design, or how people use them. A dataset can accurately record past decisions while still carrying unfair treatment into a new system. Check this: Pick one automated decision and check who appears in its data and what its success measure actually measures.

Machine learning means learning patterns from examples. Unfair outcomes can arise from those examples, from algorithm design choices, and from interactions with users. The survey describes these connected sources rather than treating unfairness as a problem with data alone. [1] Historical bias means existing social inequalities entering the data, even when collection accurately represents the world. In a hypothetical hiring case, suppose an audit establishes that past decisions rejected equally qualified women because of gender. A system trained to copy those decisions could repeat that treatment. A finding that women received only 15% of past hires would not, by itself, establish discrimination or explain its cause.

Measurement bias arises when the information selected or measured poorly represents the intended concept. A proxy is an indirect measure used in place of what we actually want to know. Consider a hypothetical healthcare system that uses spending as a proxy for health needs. If equally ill patients have different access to care, their spending could differ because of access; prioritizing higher spending could then disadvantage patients with less access. This illustrates a possible mechanism, not a healthcare finding established by the supplied excerpt.

Representation bias occurs when collected examples miss or underrepresent parts of the intended population. Overall accuracy combines results across everyone, so it can hide weaker performance for a subgroup, meaning a particular set of people within that population. In a hypothetical facial recognition test, a system that identifies people from face images might score higher overall but lower for an underrepresented group. Choosing it over a system with better results for that group would prioritize overall performance over subgroup performance; dataset size alone cannot establish that tradeoff.

For another hypothetical design choice, a system screening loan applications might be optimized only for total correct decisions, even if it makes uneven errors across groups. Comparing missed eligible applications and mistaken approvals separately can expose this tradeoff. This is an illustration of choosing what to optimize, not a finding about a particular deployed system.

A feedback loop occurs when a system's outputs affect the data used for future decisions. The survey illustrates this with search rankings: placing a result near the top can attract clicks, and using those clicks to rank future results can reinforce its visibility. Clicks therefore reflect exposure as well as user interest. [1]

As practical guidance, inspect representation, check what measurements and recorded decisions mean, compare errors across relevant groups, and monitor outcomes after deployment. These checks require a clear intended population and a stated fairness goal. Balanced group counts cannot guarantee fairness: aggregation bias, which involves drawing misleading conclusions about individuals from combined population data, can persist even with equal group representation. [1]

• Survey framework: the revised January 2022 preprint distinguishes bias arising from data from bias arising from algorithms. It also describes a feedback loop connecting data, algorithms, and user interactions; this is a conceptual organization of the literature, not a finding that every system follows the same causal pathway. [1]
• Empirical evidence summarized by the survey: research on the IJB-A and Adience facial analysis datasets found predominantly light-skinned subjects. The excerpt also explains that examining skin color and gender together can expose performance disparities hidden by broader group summaries. [1]
• Conceptual limitation of balancing data: the aggregation-bias discussion explains that ignoring relevant differences between groups can make a model unsuitable for some groups even when they have equal representation in training data. Equal group counts therefore do not establish that the model is appropriate for everyone. [1]

The supplied excerpt does not establish the hypothetical hiring or healthcare cases as empirical findings. An observed hiring proportion alone does not identify discrimination or its cause. The excerpt also does not support a general rule that synthetic or balanced data is fairer than historical data. The suggested checks are practical guidance based on the survey's categories, not a demonstrated cure; the excerpt is truncated and cannot support claims about findings beyond the supplied text.