A single Job runs two notebooks as two separate tasks. A data engineer has noticed that one of the notebooks is running slowly in the Job’s current run. The data engineer asks a tech lead for help in identifying why this might be the case.
Which of the following approaches can the tech lead use to identify why the notebook is running slowly as part of the Job?
The best way for the tech lead to identify why a notebook is running slowly as part of a Job is to navigate to the Runs tab in the Jobs UI and click on the active run to review the processing notebook. This approach enables access to detailed information about the job run, including logs, execution duration, resource utilization, and any error messages or warnings associated with the tasks. This information is essential for diagnosing performance issues.
The job run details page contains job output and links to logs, including information about the success or failure of each task in the job run. You can access job run details from the Runs tab for the job. To view job run details from the Runs tab, click the link for the run in the Start time column in the runs list view. To return to the Runs tab for the job, click the Job ID value. If the job contains multiple tasks, click a task to view task run details, including: the cluster that ran the task the Spark UI for the task logs for the task metrics for the task https://docs.databricks.com/en/workflows/jobs/monitor-job-runs.html#job-run-details
C In the Databricks Jobs UI, the Runs tab provides detailed information about the execution of each run in a Job. By clicking on the active run associated with the notebook running slowly, you can access the specific run details, including the notebook execution logs, execution duration, resource utilization, and any error messages or warnings.
"Job runs" tab
C is the correct answer. See link https://docs.databricks.com/workflows/jobs/jobs.html
B. They can navigate to the Tasks tab in the Jobs UI and click on the active run to review the processing notebook. The Tasks tab in the Jobs UI provides detailed information about each task in the job, including the task's execution time, the task's logs, and the task's output. By clicking on the active run for the notebook that is running slowly, the tech lead can review the task's logs and output to identify any issues that might be causing the slowdown. The Runs tab provides an overview of all runs of the job, but it does not provide detailed information about each task in the job.
Wrong answer. Please see documentation and you will realize the correct answer is C https://docs.databricks.com/workflows/jobs/jobs.html
c is correct
The tech lead can navigate to the Runs tab in the Jobs UI and click on the active run to review the processing notebook (Option C). This will allow them to inspect the details of the job run, including the duration of each task, which can help identify potential performance issues. There could be several reasons why a notebook is running slowly as part of a job. For instance, there might be a delay when the job cluster has to be spun up, or the table gets delta cached in memory and copies of files will be stored on local node’s storage. Even certain operations like pandas UDFs can be slow. Please note that the exact process may vary depending on the specific configurations and permissions set up in your workspace. It’s always a good idea to consult with your organization’s IT or data governance team to ensure the correct procedures are followed.
C is correct answer as we monitor job and performance of task in same way in my current project . Task tab to add another task or edit existing one
C is correct.