A machine learning engineer is using the following code block as part of a batch deployment pipeline:
Which of the following changes needs to be made so this code block will work when the inference table is a stream source?
A machine learning engineer is using the following code block as part of a batch deployment pipeline:
Which of the following changes needs to be made so this code block will work when the inference table is a stream source?
To work with a stream source in Apache Spark, you need to use the 'spark.readStream' method instead of 'spark.read'. This ensures that the DataFrame is being read as a streaming DataFrame, which is necessary for handling streaming data sources such as a Delta table that is continuously receiving new data.
C. Replace spark.read with spark.readStream
This change ensures that the DataFrame is being read as a streaming DataFrame from the stream source.
Agree with trendy01