A Databricks job has been configured with 3 tasks, each of which is a Databricks notebook. Task A does not depend on other tasks. Tasks B and C run in parallel, with each having a serial dependency on task A.
If tasks A and B complete successfully but task C fails during a scheduled run, which statement describes the resulting state?
All logic expressed in the notebook associated with tasks A and B will have been successfully completed, with some operations in task C potentially completing before it fails. In this scenario, individual tasks execute until failure, and changes made by successful tasks are retained. However, changes in the failing task depend on the point of failure, meaning some operations within task C could have been completed before the failure occurred.
Should be 'A' only, as ACID compliance is applicable at operation level. For example if task C is having 3 target delta table writes (in independent Notebook cells) then it could have after 1 write the task fails during 2nd write. In that case 1st write will still be persisted. The ACID compliance will be applicable for only the 2nd write.
A - for sure this is NOT ACID operations
Correct answer should be B as Databricks is ACID compliant
What if an operation of C is to delete a file, will the file be created after a roll back?