A machine learning engineering team wants to build a continuous pipeline for data preparation of a machine learning application. The team would like the data to be fully processed and made ready for inference in a series of equal-sized batches.
Which of the following tools can be used to provide this type of continuous processing?
Structured Streaming allows for continuous processing of data streams in real-time and outputs in equal-sized batches. This makes it suitable for building a continuous pipeline for data preparation in a machine learning application. It ensures that data is fully processed and made ready for inference efficiently.
Structured Streaming in Spark allows for continuous processing of data streams, where data can be processed in real-time and output in equal-sized batches. This makes it suitable for building continuous pipelines for data preparation in machine learning applications.
B. Structured streaming
B. Structured streaming