Azure Databricks Table Profiling for Null Value Tracking
Short answer
C Create a time series profile to evaluate null values repeatedly and observe changes over the progression of the dataset.
Question
You need to profile a table to meet the following requirements:
• The count of null values per column must be evaluated repeatedly as new records are added to the table.
• Changes in the count of null values must be observable over the progression of the dataset.
Which type of profile should you create?
Key Takeaways
The key distinction is between static snapshots and profiles that track changes over time, with 'time series' being the intended answer for observing progression despite missing explicit timestamp constraints in the prompt.
This question examines the specific capabilities of Azure Databricks Unity Catalog table profiling. It determines which profile type supports repeated evaluation and tracking of null value changes over time.
Common mistake: Candidates often select 'snapshot' because a snapshot provides an immediate count of nulls, failing to recognize that the requirement for 'repeated' evaluation and observing 'progression' implies a temporal dimension.
Expert Analysis
Why the Answer Is Correct
The correct answer is C (time series). The question requires evaluating null counts repeatedly as new records are added and observing changes over the dataset's progression. A time series profile is designed to track metrics across different points in time or data versions, allowing you to see how statistics evolve.
Why the Other Options Are Wrong
Option A (snapshot) captures the state of the table at a single point in time. While it can count nulls, it does not inherently support tracking changes over time or comparing successive states without manual intervention. Option B (inference) is used for schema discovery and data type inference, not for statistical quality monitoring like null counting.
Community Comment Notes
As user zxc01 noted, there is ambiguity regarding whether a timestamp column is required for a time series profile. They suggested that while time series is the best fit for the 'progression' requirement, Microsoft should clarify if a TIMESTAMP column is mandatory. This highlights a common exam pattern where functional requirements override strict technical prerequisites in the scenario description.
Exam Strategy
Focus on keywords indicating change over time ('progression', 'repeatedly', 'observable changes'). If the question asks for a comparison of metrics across data updates, consider time-series or versioning features rather than static snapshots.
Frequently Asked Questions
Why isn't a snapshot profile sufficient for this task?
Does a time series profile require a TIMESTAMP column?
Discussion 1 contributions
Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.
From public community threads 1
-
zxc01 👍 1 Answered CTimeSeries need column with data type TIMESTAMP but question doesn't say it. I am not sure the correct answer is A or C. Microsoft is better to add "table has TIMESTAMP data type column or table size is larger than 4 TB. However, timeseries is possible better solution if we enable change data feed (CDF) on table.
Reader comments & corrections
No reader corrections yet — spotted an error or have a note? Share it below.