A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table.
The cade block used by the data engineer is below:
If the data engineer only wants the query to execute a micro-batch to process data every 5 seconds, which of the following lines of code should the data engineer use to fill in the blank?
The correct line of code to fill in the blank to execute a micro-batch to process data every 5 seconds is: D. trigger(processingTime="5 seconds") Option A ("trigger("5 seconds")") would not work because it does not specify that the trigger should be a processing time trigger, which is necessary to trigger a micro-batch processing at regular intervals. Option B ("trigger()") would not work because it would use the default trigger, which is not a processing time trigger. Option C ("trigger(once="5 seconds")") would not work because it would only trigger the query once, not at regular intervals. Option E ("trigger(continuous="5 seconds")") would not work because it would trigger the query to run continuously, without any pauses in between, which is not what the data engineer wants.
D is the correct answer
Option D
# ProcessingTime trigger with two-seconds micro-batch interval df.writeStream \ .format("console") \ .trigger(processingTime='2 seconds') \ .start() https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#triggers
D os correct
correct syntax is D
Correct: D
https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#triggers
D val query = sourceTable .writeStream .format("delta") .outputMode("append") .trigger(Trigger.ProcessingTime("5 seconds")) .start(destinationTable)
This is Scala example. Exam should be 100% on Python.