Machine learning (ML) models are not static; their performance can degrade over time in a production environment due to factors like concept drift (changes in the underlying relationships between input and output variables) and data drift (changes in the statistical properties of the input data). A one-time review prior to deployment is critically insufficient as it fails to account for this degradation. This lack of ongoing monitoring represents a significant control gap, as an unmonitored model could produce increasingly inaccurate or biased results, leading to poor business decisions and operational risk. Therefore, this schedule is of the greatest concern to an auditor.

A: Reviewing after platform changes is a necessary control to manage "system drift" and ensure the model's performance is not negatively impacted by infrastructure updates.

B: Reviewing after functionality changes is a prudent control to ensure the model remains aligned with the business context and its intended purpose.

D: An annual review constitutes a form of ongoing monitoring. While the frequency might be insufficient for some models, it is a far superior control to no post-deployment review at all.

1. ISACA. (2021). Auditing Artificial Intelligence. The AI auditing framework describes a lifecycle that includes a "Monitor and Evaluate" phase

stating

"The performance of the AI solution should be monitored on an ongoing basis to ensure that it is operating as intended." A one-time review (Option C) completely omits this critical

ongoing phase.

2. Stanford University. (2021). CS329S: Machine Learning Systems Design

Lecture 8: Data and Model Monitoring. The courseware emphasizes that "the world is not stationary" and details the necessity of monitoring for drift (concept drift

data drift). It explicitly states that models must be continuously monitored post-deployment

making a one-time check a major deficiency. (Available via Stanford's public course materials).

3. Baylor

D.

et al. (2017). TFX: A TensorFlow-Based Production-Scale Machine Learning Platform. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. This paper from Google engineers describes a production ML pipeline where continuous monitoring and validation are core components

highlighting the industry best practice of ongoing performance analysis rather than a single pre-production check. (DOI: https://doi.org/10.1145/3097983.3098021).