The bias was introduced because the historical "nationwide information from the last 20 years" represents a skewed sample. This type of data often reflects long-standing societal biases, such as disproportionate rates of arrest, diagnosis, or reporting for drug and alcohol use among minority populations. When the AI system is trained on this data, which is not representative of the true prevalence of the disorder across all demographics, it learns and codifies these existing prejudices. The model then perpetuates this historical bias by creating plans that unfairly disadvantage minorities, mistaking the skewed data for objective reality.

A. Size disparity: While the dataset might have an unequal number of examples for different groups, the fundamental issue is that the sample itself is unrepresentative of reality, not just imbalanced.

C. Proxy data: The problem stems from the entire dataset being a biased reflection of the world, not from the specific use of one variable as a substitute for a protected characteristic.

D. Limited features: The issue is the biased quality and unrepresentative nature of the data used for training, not a lack of sufficient variables or attributes within the dataset.

1. Barocas, S., & Selbst, A. D. (2016). Big Data's Disparate Impact. California Law Review, 104(3), 671-732. In Section II.A, "Skewed Sample" (p. 677), the authors explain that training data is often a "skewed sample of the world" because it reflects existing patterns of inequality and discrimination, which a model will then learn and reproduce.

2. Corbett-Davies, S., & Goel, S. (2018). The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning. Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. Section 2.1 discusses how sampling bias occurs when the training population is not representative of the target population, leading to biased models. Historical data on policing and justice is a classic example of this phenomenon. DOI: https://doi.org/10.1145/3278721.3278728

3. Mitchell, S., et al. (2021). Algorithmic Fairness: Choices, Assumptions, and Definitions. Annual Review of Statistics and Its Application, 8, 141-163. Section 3, "Sources of Harm," details how historical bias embedded in data is a primary source of algorithmic unfairness, as models trained on such data will replicate past discriminatory patterns. DOI: https://doi.org/10.1146/annurev-statistics-042720-125902