This solution meets the requirements with the least operational overhead by using fully managed AWS services designed for this specific use case. Amazon Kinesis Data Streams is built for ingesting high-volume, real-time data. Amazon Managed Service for Apache Flink is a serverless, real-time processing service that directly integrates with Kinesis. Crucially, it includes the built-in RANDOMCUTFOREST function, which is an unsupervised algorithm specifically for detecting anomalies in streaming data. This combination provides a scalable, real-time anomaly detection pipeline without the need to provision servers, manage clusters, or develop, train, and deploy a separate machine learning model.

B: This approach introduces higher operational overhead by requiring the user to build, train, deploy, and manage a separate Amazon SageMaker endpoint, in addition to developing and managing the AWS Lambda integration code.

C: This option has the highest operational overhead because it involves self-managing an Apache Kafka cluster on Amazon EC2 instances, which requires significant effort for setup, scaling, patching, and maintenance.

D: This solution is incorrect because it uses AWS Glue for batch processing, which does not meet the real-time requirement. Amazon SQS is also less suited for real-time streaming analytics than Kinesis.

1. Amazon Managed Service for Apache Flink

RANDOMCUTFOREST: "The RANDOMCUTFOREST function assigns an anomaly score to each record. Low scores indicate that the record is normal

and high scores indicate the presence of an anomaly in the data." This built-in function is designed for this exact purpose.

Source: AWS Documentation

Amazon Managed Service for Apache Flink Developer Guide

"RANDOMCUTFOREST".

URL: https://docs.aws.amazon.com/managed-flink/latest/apiv2/analytics-sql-reference-random-cut-forest.html

2. Amazon Kinesis Data Streams for Real-Time Processing: "You can use Kinesis Data Streams to collect and process large streams of data records in real time... The processed records can then be sent to dashboards

used to generate alerts

dynamically change pricing and advertising strategies

or send data to a variety of other AWS services."

Source: AWS Documentation

Amazon Kinesis Data Streams Developer Guide

"What Is Amazon Kinesis Data Streams?".

URL: https://docs.aws.amazon.com/streams/latest/dev/introduction.html

3. Operational Overhead of Self-Managed vs. Managed Services: Managed services like Kinesis and Managed Flink reduce operational burden compared to self-managing infrastructure like Apache Kafka on EC2. "Amazon Kinesis is fully managed and runs on a serverless architecture... With Apache Kafka

you are responsible for provisioning servers

managing cluster capacity..."

Source: AWS Documentation

Streaming Data Solutions on AWS with Amazon Kinesis

"Amazon Kinesis versus Apache Kafka".

URL: https://aws.amazon.com/kinesis/streaming-data/kinesis-vs-kafka/