Amazon SageMaker Experiments is the purpose-built service for organizing, tracking, comparing, and evaluating machine learning experiments. The developer can create an experiment and then run the PySpark script as a SageMaker Processing Job. For each run (trial), they can use the Experiments SDK to log the specific feature transformation parameters and sample sizes used, along with the resulting model accuracy and inference performance metrics. This provides a structured and systematic way to analyze how different preprocessing configurations impact model outcomes, directly fulfilling the requirements of the scenario.

A. SageMaker Debugger is for monitoring and debugging training jobs in real-time (e.g., checking for vanishing gradients), not for systematically comparing the results of different experimental runs.

C. SageMaker Autopilot automates the entire ML workflow. It does not provide a framework for a developer to manually and systematically test their own custom PySpark feature transformations.

D. SageMaker Model Monitor is a post-deployment tool used to detect data and model quality drift in production, not for tracking and comparing experiments during the development phase.

1. Amazon SageMaker Developer Guide - Amazon SageMaker Experiments: "Amazon SageMaker Experiments lets you organize, track, compare, and evaluate your machine learning (ML) experiments... You can track the inputs (data, parameters, and algorithms) and outputs (metrics and model artifacts) of your experiments as trials."

2. Amazon SageMaker Developer Guide - Track Experiments with the Python SDK: This section details how to use sagemaker.experiments.Run to log parameters and metrics, which is the exact mechanism needed. It states, "You can log any information that helps you evaluate your experiment."

3. Amazon SageMaker Developer Guide - Monitor and Analyze Training Jobs Using Metrics: This page describes SageMaker Debugger's purpose: "Use Amazon SageMaker Debugger to debug, monitor, and profile your machine learning training jobs," which is distinct from experiment tracking.

4. Amazon SageMaker Developer Guide - What Is Amazon SageMaker Model Monitor?: This document clarifies Model Monitor's role: "Amazon SageMaker Model Monitor helps you monitor machine learning (ML) models in production and notifies you when quality issues arise." This confirms it is a post-deployment tool.