Do Algorithms Have Bias?

Peter Bruce

July 6, 2018

BLOG_Do Algorithms Have Bias

Algorithmic bias is a popular topic; see for example this article describing how Microsoft is working on a dashboard product to detect unfair bias in algorithms. When a typical person (not a statistician) uses the term "bias" they usually have in mind unfair prejudgment, or stacking of the deck, against a person based on some aspect of that person's identity (race, gender, ethnic background, religion, nationality, etc.). Until recently, "bias" meant something very different to statisticians.

statistical bias example plot-2Statistical bias is the tendency of an estimator, algorithm or other system (could be a physical system like a gun) to miss a target systematically in a particular direction (e.g., in the plot on the right, nearly always hitting above the negatively sloped diagonal, where the target is 0,0).

One example of statistical bias is the formula for variance. Suppose you have a randomly-drawn sample of homes. Common sense tells us that the average home value in the sample is a good estimate of the average home value in the larger population. One sample might be too low, another too high, due to the luck of the draw, but, on balance, the average in the sample will tend towards the average in the larger population. And this is true of the average (mean). However, it is not true for the variance (average of squared deviations from the mean). The sample variance will systematically underestimate the true variance of the population - it is biased. Only if you divide the sum of squared deviations by n-1 will the sample variance be unbiased.

Of course, this definition has nothing to do with "fairness."

Matthias SpielkampNow, the popular conception of bias as involving unfairness to a group of people based on identity is increasingly making its way into the field of statistics and machine learning; consider this article by Matthias Spielkamp (left) published in the MIT Technology Review: Inspecting Algorithms for Bias.

This sort of algorithmic bias falls into several categories:

    1. One category is what the courts term "disparate impact" where an algorithm produces different results for one group compared to another, but the algorithm is not explicitly basing decisions on group identity, and results end up differing by group for causes that are inextricably to other factors. For example, home ownership rates are linked to neighborhood crime rates, and hispanics have lower home ownership rates, so any algorithm making predictions (say, for burglary insurance) on the basis of homeownership will appear to discriminate against hispanics though the predictions of exposure to burglary may be accurate.
    2. A related category is where an algorithm may not explicitly take group membership into account, and yet the predictions end up being inaccurate in ways that are unfair to certain groups. ProPublica analyzed the COMPAS system for predicting recidivism (commission of additional crimes after conviction for an initial crime) and found that it over-identified blacks as recidivist.
    3. A third category is the "black-box" problem - where a machine learning system is used that masks the roles played by predictors so you may not know whether or how group membership plays a role in the predictions. As with categories 1 and 2, predictions might be accurate or inaccurate, but the lack of explainability lowers confidence, and hampers further analysis and improvement of the algorithm
    4. A fourth category of algorithmic bias in supervised predictive modeling is if the training data has been labelled by humans whose judgment is biased, and so bias is built in to the models from the beginning.  This is less of a problem when the label is fact-based (the batter got a hit or not), rather than judgement based (how elegantly the ballerina danced) [1].Book cover Weapons of Math Destruction

Cathy O'Neil has an engaging discussion of these and related issues in her provocative book Weapons of Math Destruction.

Need help getting started with analytics? Our on-site half-day Analytics Executive Strategy session delivers strategies for using analytics to improve organizational decision-making, recommendations on how to grow your analytics capabilities, and plans for short and long-term analytics opportunities, prioritized based on feasibility and return on investment. Learn more. 

[1] Editor's Note: This bias was suggested by Dr. John Elder.


Read part two of this blog series Picking Favorites: A Brief Introduction to Selection Bias

Download the eBook Top 10 Data Mining Mistakes.

Read the blog Avoid Reinforcement Bias When Fishing in the Same Pond.

About the Author

Peter Bruce Peter Bruce is Founder and President of The Institute for Statistics Education at Previously he taught statistics at the University of Maryland, and served in the U.S. Foreign Service. He is a co-author of Data Mining for Business Analytics, with Galit Shmueli and Nitin R. Patel (Wiley, 3rd ed. 2016; also JMP version 2017, R version 2018, Python version 2019; plus translations into Korean and, forthcoming, Chinese), Introductory Statistics and Analytics (Wiley, 2015), and Practical Statistics for Data Scientists, with Andrew Bruce, (O'Reilly 2016). His blogs on statistics are featured regularly in Scientific American online.