The requirement is to process data in micro-batches with a fixed interval of 5 seconds. The trigger(processingTime='5 seconds') configuration is specifically designed for this purpose. It instructs the Structured Streaming engine to initiate a new micro-batch at the specified interval of processing time. This ensures that data is processed periodically every 5 seconds, which directly fulfills the engineer's requirement for a fixed-interval micro-batching system.

A: trigger(continuous='5 seconds') configures continuous processing mode, which aims for millisecond latency, not fixed-interval micro-batches. The interval here is for checkpointing, not for batch triggering.

B: trigger() is not valid syntax as the method requires an argument. The default behavior (if .trigger() is omitted) processes a new micro-batch as soon as the previous one finishes, not at a fixed interval.

D: trigger(processingTime=5000) is syntactically incorrect. The processingTime parameter expects a string representing the time interval (e.g., '5 seconds'), not an integer representing milliseconds.

---

1. Apache Spark Official Documentation: The Structured Streaming Programming Guide explicitly defines the processingTime trigger.

Source: Apache Spark 3.5.1 Documentation

Structured Streaming Programming Guide

Section: Triggers.

Details: The documentation states

"ProcessingTime trigger: The query will be executed in micro-batch mode

where micro-batches will be kicked off at the user-specified intervals. If processingTime is 0

the query will be executed as fast as possible." It provides the syntax trigger(processingTime='5 seconds'). This directly supports option C and refutes D. It also describes the continuous trigger

confirming why A is incorrect for this scenario.

2. Databricks Official Documentation: The Databricks documentation on Structured Streaming triggers reinforces this concept.

Source: Databricks Documentation

"Configure Structured Streaming triggers".

Details: The section on "Processing time trigger" explains: "If you specify a processing time interval

Structured Streaming will trigger a micro-batch at that interval." It shows the example df.writeStream.trigger(processingTime='10 seconds')

which validates the syntax and functionality described in option C.