Delta Retention Properties for Auditing Deleted Data
You have an Azure Databricks workspace that is enabled for Unity Catalog and contains a managed Delta table named Table1. Table1 stores customer data. You need to implement a data retention solution that meets the following requirements: Deleted data must be retained for 30 days to support audits. Deleted data that is older than 30 days must be removed permanently. The solution must minimize administrative effort Which two properties should you configure? Each correct answer presents part of the solution. NOTE: Each correct selection is worth one point.
Community Insight
The question tests whether you know that Delta retains deleted data in two distinct places—the transaction log and the soft-deleted files still on disk—and that both retention clocks must be set to 30 days; the trap is assuming one property (or deletion vectors) is enough.
In a Unity Catalog–enabled Databricks workspace, the audit-friendly retention of deleted data in a managed Delta table such as Table1 is controlled by two table properties: delta.logRetentionDuration and delta.deletedFileRetentionDuration. Configuring both to a 30-day interval keeps transaction history and soft-deleted files available for exactly 30 days and lets VACUUM permanently purge anything older, with no extra tooling required.
The most common wrong pick is the BD combination (delta.enableDeletionVectors + delta.deletedFileRetentionDuration). Learners argue deletion vectors keep rows readable forever, but disabling or enabling deletion vectors changes table behavior rather than defining a bounded, automatic retention window, and it conflicts with the requirement to permanently remove data after 30 days.
Community Discussion (4 comments)
Comments & Corrections
No comments yet — spotted an error or have a note? Share it below.
Expert Analysis
Why the Answer Is Correct
delta.deletedFileRetentionDuration governs how long files deleted by MERGE/DELETE/UPDATE or overwrite operations remain on storage before VACUUM considers them eligible for permanent removal (default is 7 days, not 30), so it must be raised to 'interval 30 days' to meet the audit requirement. delta.logRetentionDuration controls how long the transaction history (the Delta _delta_log entries) is preserved before checkpoint compaction cleans it up; its default is 30 days, and explicitly setting it to 'interval 30 days' guarantees the audit trail for the same window. Together these two properties give a 30-day audit window for both the data and its metadata history, after which VACUUM automatically purges the files—no additional tooling or scripting needed, which satisfies the minimize-administrative-effort requirement.
Why the Other Options Are Wrong
delta.enableDeletionVectors (B) switches DML operations to merge-on-read mode that marks rows as deleted instead of rewriting files; it does not create a bounded 30-day retention period, and turning it off merely reverts to copy-on-write behavior without defining any retention policy. delta.timeUntilArchived (A) is an archival property that moves old history versions to archive storage, which is a heavier, storage-consuming mechanism well beyond a simple 30-day retention requirement. delta.autoOptimize.autoCompact (E) is a performance setting that compacts small files during writes and has no effect on how long deleted data survives or when it is purged.
Community Comment Notes
Abhi_007's top-voted comment correctly explains that delta.logRetentionDuration keeps transaction history for 30 days and delta.deletedFileRetentionDuration must be raised from its 7-day default to 30 days for the audit requirement, yielding the CD answer. mtc9 agrees on CD while correctly noting that enableDeletionVectors "doesn't affect retention of deleted files." In dissent, zxc01 argues for BD, claiming that with deletion vectors "you still can query deleted data" via time travel and that deletion vectors keep marked files indefinitely—but that produces an unbounded, non-automatic retention that never "must be removed permanently" without manual REORG TABLE work, so it fails the stated constraints.
Official Reference
Exam Strategy
On retention questions, map each requirement to a mechanism: audit window over deleted data means soft-deleted file retention, and audit of change history means log retention. Set both Delta table properties to the same interval and note that the log default is already 30 days while the file default is only 7 days.
Frequently Asked Questions
Why isn't delta.deletedFileRetentionDuration alone enough to meet the requirement?
The transaction log (_delta_log) is a separate retention track with its own 30-day clock. Without delta.logRetentionDuration, the history of the deletions can age out independently, breaking the audit trail for the deleted customer data.
Why is delta.enableDeletionVectors wrong here even though deleted rows stay readable?
Deletion vectors mark rows as deleted but keep the underlying files indefinitely, so nothing is removed permanently without a manual REORG TABLE step. That fails the automatic 30-day permanent-removal requirement and increases administrative effort.