| Type | Computer science concept |
| Outcome | Reinforces societal biases |
| Examples | Job applicant screening • Credit scoring • Criminal justice system • Online advertising |
| Main issues | Inadvertent discrimination • Reinforced inequalities • Inaccurate predictions • Inappropriate decision-making |
| Place of origin | |
| Related concepts | Fairness, accountability, and transparency in machine learning • AI ethics • Bias in machine learning |
| Conceptualization | Mid-1990s |
| Mitigation strategies | Data pre-processing • Algorithm modification • Post-processing • Regular audits |
Algorithmic Bias is a phenomenon in which computer programs used for data analysis, predictive modeling, and automated decision-making systems inadvertently reproduce and/or reinforce historically present societal inequalities and biases, often having discriminatory effects based on race, gender, age, disability, and other individually identifying characteristics.
The origins of algorithmic bias can be traced back to the mid-1990s, as the proliferation of Internet usage and the growing availability of large datasets caught the attention of computer scientists seeking to extract insights for commercial, government, and military purposes. The concept became known as machine learning, a subset of artificial intelligence that employs statistical models to infer patterns and make predictions from vast datasets, often referred to as big data.
An early example of algorithmic bias occurred in 2006 when Amazon inadvertently introduced gender bias into their hiring algorithm. The program was found to have favored male candidates, as it was trained primarily on historical resumes submitted to the company by men, mirroring existing gender disparities in the tech sector. This revelation prompted a public outcry and caused Amazon to discontinue the use of the algorithm.
In the following years, a series of high-profile controversies brought the issue of algorithmic bias to the forefront, igniting public concern and scrutiny. In 2010, Harvard University researcher Latanya Sweeney uncovered racial bias in search engine results that displayed higher numbers of arrest-related advertisements when searching for names commonly associated with African Americans. Sweeney's work alerted the public to the potential for biases hiding in seemingly neutral computer systems.
In 2014, COMPAS, a risk assessment algorithm used by court systems across the US to aid in sentencing, was found to have over predicted recidivism rates for black defendants compared to their white counterparts. This discovery led to questions around the Fair Housing Act, as well as the broader legal implications of biased algorithms operating in the criminal justice system.
As awareness of algorithmic bias grew, academics, researchers, and advocacy groups began calling for increased transparency and scrutiny of the algorithms used by companies and governments. Activists urged for the development of novel auditing, testing, and accountability mechanisms to ensure algorithmic fairness and prevent discrimination.
In 2018, the European Union General Data Protection Regulation (GDPR) enacted new right to explanation legislation, which required that any decision made by an automated system can be explained to the subject it affects. The GDPR guidelines have had a far-reaching impact on algorithmic transparency and pushed towards the development of more interpretable, less biased algorithms.
The United States has also debated various policy proposals to address algorithmic bias. Algorithmic Accountability Acts introduced in Congress have aimed to regulate automated decision systems used in hiring, credit, and lending, requiring companies to audit, test, and correct their systems for biases. Additionally, in 2020, several states, including California, enacted state-level Algorithmic Fairness Act legislation to extend accountability to a broader set of automated systems and decisions.
Despite growing attention to the issue of algorithmic bias, challenges persist as artificial intelligence becomes increasingly integrated into daily life. Stakeholders from government, industry, academia and civil society will need to collaborate to craft solutions that address algorithmic bias from multiple angles.
Diverse pipeline solutions to ensure interdisciplinary perspectives in the technology sector, algorithmic auditing solutions to routinely test systems for biases, and ethical reporting solutions to openly disclose activities are essential to combating algorithmic bias. Public education and engagement with the ethical implications of artificial intelligence remain crucial to sustaining momentum towards fair and equitable algorithmic decision-making.
