Azure Databricks Table Profiling for Null Value Tracking
You have an Azure Databricks workspace that is enabled for Unity Catalog. You need to profile a table to meet the following requirements: • The count of null values per column must be evaluated repeatedly as new records are added to the table. • Changes in the count of null values must be observable over the progression of the dataset. Which type of profile should you create?
Community Insight
The key distinction is between static snapshots and profiles that track changes over time, with 'time series' being the intended answer for observing progression despite missing explicit timestamp constraints in the prompt.
This question examines the specific capabilities of Azure Databricks Unity Catalog table profiling. It determines which profile type supports repeated evaluation and tracking of null value changes over time.
Candidates often select 'snapshot' because a snapshot provides an immediate count of nulls, failing to recognize that the requirement for 'repeated' evaluation and observing 'progression' implies a temporal dimension.
Community Discussion (1 comments)
Comments & Corrections
No comments yet — spotted an error or have a note? Share it below.
Expert Analysis
Why the Answer Is Correct
The correct answer is C (time series). The question requires evaluating null counts repeatedly as new records are added and observing changes over the dataset's progression. A time series profile is designed to track metrics across different points in time or data versions, allowing you to see how statistics evolve.Why the Other Options Are Wrong
Option A (snapshot) captures the state of the table at a single point in time. While it can count nulls, it does not inherently support tracking changes over time or comparing successive states without manual intervention. Option B (inference) is used for schema discovery and data type inference, not for statistical quality monitoring like null counting.Community Comment Notes
As user zxc01 noted, there is ambiguity regarding whether a timestamp column is required for a time series profile. They suggested that while time series is the best fit for the 'progression' requirement, Microsoft should clarify if a TIMESTAMP column is mandatory. This highlights a common exam pattern where functional requirements override strict technical prerequisites in the scenario description.Exam Strategy
Focus on keywords indicating change over time ('progression', 'repeatedly', 'observable changes'). If the question asks for a comparison of metrics across data updates, consider time-series or versioning features rather than static snapshots.
Frequently Asked Questions
Why isn't a snapshot profile sufficient for this task?
A snapshot only provides a static view of data at one moment. It cannot inherently track or display changes over time as new records are added.
Does a time series profile require a TIMESTAMP column?
While ideal, the exam scenario focuses on the functional need to track progression. In practice, time series profiles leverage versioning or timestamps to show metric evolution.