Certified Data Engineer Professional Exam

Exam Certified Data Engineer Professional All QuestionsBrowse all questions from this exam

Question 28

A junior data engineer seeks to leverage Delta Lake's Change Data Feed functionality to create a Type 1 table representing all of the values that have ever been valid for all rows in a bronze table created with the property delta.enableChangeDataFeed = true. They plan to execute the following code as a daily job:

Which statement describes the execution and results of running the above query multiple times?

Each time the job is executed, newly updated records will be merged into the target table, overwriting previous values with the same primary keys.

Each time the job is executed, the entire available history of inserted or updated records will be appended to the target table, resulting in many duplicate entries.

Each time the job is executed, the target table will be overwritten using the entire history of inserted or updated records, giving the desired result.

Each time the job is executed, the differences between the original and current versions are calculated; this may result in duplicate entries for some records.

Each time the job is executed, only those records that have been inserted or updated since the last execution will be appended to the target table, giving the desired result.

Correct Answer: B

The provided code uses Delta Lake's Change Data Feed (CDF) functionality to read and filter changes from the beginning (startingVersion, 0) each time it runs. It filters for update postimage and insert changes only and appends the filtered results to the target table. Due to starting from version 0 every time, the entire history of inserted or updated records will be read and appended to the target table on each execution, leading to many duplicate entries.

Discussion

imatheushenriqueOption: B

("startingVersion", 0) that means the entiry history of table will be read so B.

PrashantTiwariOption: B

B is correct

kz_dataOption: B

B is correct

azurearchOption: B

B is the right answer, sorry.

asmayassinegOption: B

sorry, answer is correct B.

asmayassinegOption: A

Answer is A, since the df is filtering on updated records using update_postimage filter

taif12340

it's B: Reading table’s changes, captured by CDF, using spark.read means that you are reading them as a static source. So, each time you run the query, all table’s changes (starting from the specified startingVersion) will be read.

mht3336

there is also insert in the filter.

5ffcd04Option: B

Correct B

azurelearn2020Option: B

correct answer is B.

[Removed]Option: E

Considering that we are talking about Change Data Feed and the code is filtering by[ "update_postimage", "insert" ] the column "_change_type", I would go with the option E. Reference: https://docs.delta.io/latest/delta-change-data-feed.html#:~:text=_change_type,update_preimage%20%2C%20update_postimage

5ffcd04

Notice option ("startingVersion", 0), which will bring all changes from begining. Hence Answer is B.

jyothsna12496Option: B

why is it Not E. It gets newly inserted or updated records

[Removed]

I'm with you, follow the reference: https://docs.delta.io/latest/delta-change-data-feed.html#:~:text=_change_type,update_preimage%20%2C%20update_postimage

5ffcd04

Notice .option ("startingVersion", 0), which will bring all changes from begining. Hence Answer is B.

sturcuOption: B

correct

azurearchOption: A

answer is A, because there is a filter as asmayassineg said. Filter filters only existing records from change feed