A DLT pipeline includes the following streaming tables:
• raw_iot ingests raw device measurement data from a heart rate tracking device.
• bpm_stats incrementally computes user statistics based on BPM measurements from raw_iot.
How can the data engineer configure this pipeline to be able to retain manually deleted or updated records in the raw_iot table, while recomputing the downstream table bpm_stats table when a pipeline update is run?
To retain manually deleted or updated records in the raw_iot table and ensure the downstream table bpm_stats is recomputed when a pipeline update is run, the data engineer needs to set the pipelines.reset.allowed property to false on raw_iot. This prevents the raw_iot table from being reset during pipeline updates, ensuring that the manual changes are preserved.
answer A This property, when set to false, ensures that the table will not be reset during pipeline updates, thus preserving manually deleted or updated records. This is crucial for the raw_iot table to retain the manual modifications.
Set the pipelines.reset.allowed property to false on raw_iot
Answer: B