Certified Data Engineer Associate Exam - Question 38

Question

An engineering manager wants to monitor the performance of a recent project using a Databricks SQL query. For the first week following the project’s release, the manager wants the query results to be updated every minute. However, the manager is concerned that the compute resources used for the query will be left running and cost the organization a lot of money beyond the first week of the project’s release.

Which of the following approaches can the engineering team use to ensure the query does not cost the organization any money beyond the first week of the project’s release?

Examice · Accepted Answer

To ensure the query does not cost the organization any money beyond the first week of the project’s release, the engineering team can set the query's refresh schedule to end on a specific date. By configuring the query scheduler to stop the refresh after the first week, they can prevent the compute resources from being unnecessarily consumed beyond the required period. This method effectively controls the duration for which the query runs, thereby managing and limiting the associated costs.

Nika12 · Answer

Just got 100% on the test. E was correct. C was not in the available options.

BigDaddyAus · Answer

The query scheduler only gives the option on what the interval is to run the query. It does not provide a way to stop after x iterations or at a point in time. 
The question is confusing. From what i found the only option is to limit users access to the query (and therefore query scheduler). 
https://docs.databricks.com/security/auth-authz/access-control/query-acl.html
Not convinced how this would be helping the organization save money if no-one is manually stopping the schedule. 
Answer C seems most correct
Answer D can be achieved using acl however how is this helpful in the use case described?

Garyn · Answer

E. They can set the query’s refresh schedule to end on a certain date in the query scheduler.

Explanation:

Query Scheduler: Databricks offers a Query Scheduler that allows users to schedule the execution of SQL queries at specific intervals or for specific durations.

Setting a Specific End Date: The team can configure the query's refresh schedule to conclude or end on a certain date. By specifying an end date within the first week of the project's release, the query will automatically stop refreshing after that date. This action ensures that compute resources aren't continuously utilized beyond the specified timeframe, preventing unnecessary costs.

This approach allows the team to control and limit the execution of the query to the required duration without incurring additional costs beyond the first week of the project's release.

data_arch · Answer

Answer is E
It´s true natively the query can´t be scheduled to stop, but the scheduler allow us to use cron syntax.
So we can define the year, month and days of the first week and the trigger won´t run after that

chays · Answer

agree with BigDaddyAus

LANDIS · Answer

Answer is E
https://docs.databricks.com/sql/user/queries/schedule-query.html#schedule-a-query

Inhaler_boy · Answer

Answer is C. According to documentation it cant be scheduled up until a certain date. It has to be in intervals and then canceled manually. They don't mention end date. Only start date and intervals.
https://docs.databricks.com/en/workflows/jobs/schedule-jobs.html

god_father · Answer

E is the correct answer.

From the docs:

If a dashboard is configured for automatic updates, it has a Scheduled button at the top, rather than a Schedule button. To stop automatically updating the dashboard and remove its subscriptions:

Click Scheduled.
    In the Refresh every drop-down, select Never.
    Click Save. The Scheduled button label changes to Schedule.
Source: https://learn.microsoft.com/en-us/azure/databricks/sql/user/dashboards/

Def21 · Answer

The query scheduler does not give option to have end date (or iterations). Dashboards might give one, but the question specifically mentions queries.
https://learn.microsoft.com/en-gb/azure/databricks/sql/user/queries/schedule-query

Atnafu · Answer

E
Option A:  The query will still run, but it will be throttled if it exceeds the DBU limit.
Option B:The query will still run, but it will only run a certain number of times before it stops.
Option C: The engineering team can ensure 
Option D:  The query will still run, but only the individuals who are authorized to manage the refresh schedule will be able to stop it.
E-Answer 
Therefore, the correct answer is that the engineering team can set the query’s refresh schedule to end on a certain date in the query scheduler to ensure the query does not cost the organization any money beyond the first week of the project’s release.

[Removed] · Answer

agree with BigDaddyAus

damaldon · Answer

Correct Answer E.

kishore1980 · Answer

The picker scrolls and allows you to choose:
An interval: 1-30 minutes, 1-12 hours, 1 or 30 days, 1 or 2 weeks

Since the schedule picker allows to choose interval to refresh query every 1 or 2 weeks. If we choose 1 week the schedule ends after a week. So the answer is B.

kishore1980 · Answer

Option E is correct answer

mokrani · Answer

C is the correct answer

Source : https://docs.databricks.com/en/sql/user/queries/schedule-query.html

aspix82 · Answer

Answer is E

3fbc31b · Answer

The correct answer is E for this question.

Certified Data Engineer Associate Exam - Question 38

Discussion