Scenario: A document classification model detects fraud. It performs well on the majority ("legitimate claim") documents but frequently misclassifies the minority ("fraudulent claim") samples. SageMaker Clarify pretraining bias analysis reveals a significant skew in the dataset. Question- What issue is most likely causing the model's poor performance on fraudulent claim detection? Options:

The scenario explicitly states that a "SageMaker Clarify pretraining bias analysis reveals a significant skew in the dataset." This directly points to Class Imbalance (CI), a pre-training bias metric that SageMaker Clarify is designed to detect. Class Imbalance occurs when the distribution of target labels is unequal. In this fraud detection case, "fraudulent claim" samples are the minority class. A model trained on such a dataset can achieve high accuracy by simply predicting the majority class ("legitimate claim"), failing to learn the distinguishing features of the minority class, which explains the poor performance on fraudulent claims.

A. A high learning rate is a hyperparameter issue that typically causes training instability or failure to converge, not a specific bias against a minority class.

C. Insufficient text extraction is a feature engineering problem. While it could degrade overall performance, it does not inherently explain the specific failure on the minority class versus the majority class.

D. Overfitting describes poor generalization to any unseen data. The problem here is a specific, predictable failure on the minority class due to dataset distribution, which is more precisely defined as bias from class imbalance.

1. Amazon SageMaker Developer Guide: Under the section "Pretraining Bias Metrics," the guide lists "Class Imbalance (CI)" as a key metric. It states, "A dataset is imbalanced if the class labels are not equally distributed... The CI is the ratio of the number of samples in the majority class to the number of samples in the minority class." This directly links the "significant skew" found by Clarify to the concept of Class Imbalance.

Source: Amazon SageMaker Developer Guide, Section: "Bias detection with SageMaker Clarify" -> "Pretraining Bias Metrics".

2. AWS Machine Learning Blog: In the article "Handle class imbalance with Amazon SageMaker," it is explained that "if the training dataset has a disproportionate ratio of samples in each class, a machine learning (ML) model may be biased towards the majority class. This is a common problem in use cases like fraud detection... where the number of fraudulent transactions is much smaller than the number of legitimate ones." This aligns perfectly with the scenario.

Source: AWS Machine Learning Blog, "Handle class imbalance with Amazon SageMaker," May 18, 2022.

3. Academic Publication: He, H., & Garcia, E. A. (2009). Learning from Imbalanced Data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284. This foundational paper explains that standard algorithms are often biased towards the majority class and that their performance on the minority class is compromised.

DOI: https://doi.org/10.1109/TKDE.2008.239, Section II-A, "The Nature of the Imbalance Problem".