Certified Data Engineer Professional Exam

Exam Certified Data Engineer Professional All QuestionsBrowse all questions from this exam

Question 147

A data architect has heard about Delta Lake’s built-in versioning and time travel capabilities. For auditing purposes, they have a requirement to maintain a full record of all valid street addresses as they appear in the customers table.

The architect is interested in implementing a Type 1 table, overwriting existing records with new values and relying on Delta Lake time travel to support long-term auditing. A data engineer on the project feels that a Type 2 table will provide better performance and scalability.

Which piece of information is critical to this decision?

Data corruption can occur if a query fails in a partially completed state because Type 2 tables require setting multiple fields in a single update.

Shallow clones can be combined with Type 1 tables to accelerate historic queries for long-term versioning.

Delta Lake time travel cannot be used to query previous versions of these tables because Type 1 changes modify data files in place.

Delta Lake time travel does not scale well in cost or latency to provide a long-term versioning solution.

Correct Answer: D

Delta Lake time travel does not scale well in cost or latency to provide a long-term versioning solution. While Delta Lake's built-in versioning and time travel capabilities are powerful, they can become expensive and slow when used to maintain a long-term record of historical data. A Type 1 table, which constantly overwrites records, would require frequent time travel queries for historical data, leading to potential performance and scalability issues. Therefore, knowing the limitations of Delta Lake time travel in terms of cost and latency is critical to deciding between Type 1 and Type 2 tables for this use case.

Discussion

hpkrOption: D

correct answer - D