Configuring Azure Databricks Cluster Autoscaling and Termination
You have an Azure Databricks workspace that contains an all-purpose cluster named Cluster1. You need to configure Cluster1 to meet the following requirements: Scale up automatically when workloads increase Scale down automatically when workloads decrease Minimize costs Which two actions should you perform? Each correct answer presents part of the solution. NOTE: Each correct selection is worth one point.
Community Insight
The question tests knowledge of cluster lifecycle management, specifically identifying that autoscaling handles dynamic workload changes while auto-termination prevents idle resource waste.
This question addresses how to optimize Azure Databricks cluster costs by configuring automatic scaling and inactivity termination. It establishes that enabling autoscaling and setting an auto-termination timeout are the two required actions to meet the specified requirements.
Learners often select 'Disable Photon acceleration' (Option A) thinking it reduces costs, but this is a distractor as it may increase runtime costs and does not address scaling or idle time directly.
Community Discussion (2 comments)
Comments & Corrections
No comments yet — spotted an error or have a note? Share it below.
Expert Analysis
Why the Answer Is Correct
To satisfy the requirement to scale up and down automatically, you must enable autoscaling on the cluster (Option B). This allows Azure Databricks to adjust the number of workers based on the workload load. To minimize costs associated with idle resources, you should configure the cluster to terminate after a period of inactivity, such as 30 minutes (Option E). Together, these two settings ensure efficient resource utilization and cost control.Why the Other Options Are Wrong
Option A (Disable Photon acceleration) is incorrect because disabling a performance accelerator like Photon typically slows down query execution, which can lead to longer runtimes and potentially higher costs, rather than minimizing them. Option C (Apply a compute policy) is about governance and access control, not direct cost or scaling configuration for a specific cluster instance. Option D (Specify a fixed number of workers) contradicts the requirement for automatic scaling, as it prevents the cluster from adjusting its size dynamically.Community Comment Notes
Community consensus strongly supports options B and E. One user noted that while disabling Photon might reduce DBU costs, it increases runtime, making it a poor choice for overall cost minimization compared to auto-termination. Another user highlighted that autoscaling is the only option that directly addresses the scaling requirements, confirming B and E as the correct pair.Exam Strategy
When asked to minimize costs and handle variable workloads, always look for 'Autoscaling' to handle the variability and 'Auto-termination' to handle idleness. Avoid options that disable performance features unless explicitly stated as a cost-saving measure for short-lived jobs, as they often have hidden performance penalties.
Frequently Asked Questions
Why is disabling Photon acceleration not the best way to minimize costs?
Disabling Photon can significantly slow down query execution, leading to longer runtimes and potentially higher total compute costs, offsetting any DBU savings.
Does specifying a fixed number of workers help with cost minimization?
No, fixing the worker count prevents the cluster from scaling down during low activity, leading to wasted resources and higher costs compared to using autoscaling.