Each configuration below is identical to the extent that each cluster has 400 GB total of RAM 160 total cores and only one Executor per VM.
Given an extremely long-running job for which completion must be guaranteed, which cluster configuration will be able to guarantee completion of the job in light of one or more VM failures?
Ensuring the completion of an extremely long-running job in the event of one or more VM failures requires a cluster configuration that distributes the workload across the highest number of VMs. This way, the impact of losing any single VM is minimized and the remaining VMs can continue the job. Option B, with 16 VMs, each with 25 GB per Executor and 10 cores per Executor, offers the best setup for resilience and redundancy. More VMs mean that the job can still likely proceed even if several VMs fail, ensuring the completion of the job with minimal interruption.
If VM is down, performance is degraded, so opting for vm's which has distributed memory per executor and optimal cores per executor.
in my exam today, i chose B, 16VM, because the "extremely long-run".
do you have link to databricks doc?
This setup ensures that the job can continue running and complete even if some VMs fail, as there are more VMs available to handle the workload