Delta Retention Properties for Auditing Deleted Data
Short answer
CD Set delta.logRetentionDuration and delta.deletedFileRetentionDuration to 'interval 30 days' on Table1 so deleted data and its audit history survive 30 days before VACUUM purges them.
Question
You need to implement a data retention solution that meets the following requirements:
Deleted data must be retained for 30 days to support audits.
Deleted data that is older than 30 days must be removed permanently.
The solution must minimize administrative effort
Which two properties should you configure? Each correct answer presents part of the solution.
NOTE: Each correct selection is worth one point.
Key Takeaways
The question tests whether you know that Delta retains deleted data in two distinct places—the transaction log and the soft-deleted files still on disk—and that both retention clocks must be set to 30 days; the trap is assuming one property (or deletion vectors) is enough.
In a Unity Catalog–enabled Databricks workspace, the audit-friendly retention of deleted data in a managed Delta table such as Table1 is controlled by two table properties: delta.logRetentionDuration and delta.deletedFileRetentionDuration. Configuring both to a 30-day interval keeps transaction history and soft-deleted files available for exactly 30 days and lets VACUUM permanently purge anything older, with no extra tooling required.
Common mistake: The most common wrong pick is the BD combination (delta.enableDeletionVectors + delta.deletedFileRetentionDuration). Learners argue deletion vectors keep rows readable forever, but disabling or enabling deletion vectors changes table behavior rather than defining a bounded, automatic retention window, and it conflicts with the requirement to permanently remove data after 30 days.
Expert Analysis
Why the Answer Is Correct
delta.deletedFileRetentionDuration governs how long files deleted by MERGE/DELETE/UPDATE or overwrite operations remain on storage before VACUUM considers them eligible for permanent removal (default is 7 days, not 30), so it must be raised to 'interval 30 days' to meet the audit requirement. delta.logRetentionDuration controls how long the transaction history (the Delta _delta_log entries) is preserved before checkpoint compaction cleans it up; its default is 30 days, and explicitly setting it to 'interval 30 days' guarantees the audit trail for the same window. Together these two properties give a 30-day audit window for both the data and its metadata history, after which VACUUM automatically purges the files—no additional tooling or scripting needed, which satisfies the minimize-administrative-effort requirement.
Why the Other Options Are Wrong
delta.enableDeletionVectors (B) switches DML operations to merge-on-read mode that marks rows as deleted instead of rewriting files; it does not create a bounded 30-day retention period, and turning it off merely reverts to copy-on-write behavior without defining any retention policy. delta.timeUntilArchived (A) is an archival property that moves old history versions to archive storage, which is a heavier, storage-consuming mechanism well beyond a simple 30-day retention requirement. delta.autoOptimize.autoCompact (E) is a performance setting that compacts small files during writes and has no effect on how long deleted data survives or when it is purged.
Community Comment Notes
Abhi_007's top-voted comment correctly explains that delta.logRetentionDuration keeps transaction history for 30 days and delta.deletedFileRetentionDuration must be raised from its 7-day default to 30 days for the audit requirement, yielding the CD answer. mtc9 agrees on CD while correctly noting that enableDeletionVectors "doesn't affect retention of deleted files." In dissent, zxc01 argues for BD, claiming that with deletion vectors "you still can query deleted data" via time travel and that deletion vectors keep marked files indefinitely—but that produces an unbounded, non-automatic retention that never "must be removed permanently" without manual REORG TABLE work, so it fails the stated constraints.
Official Reference
These links open the official vendor documentation in a new tab.
Exam Strategy
On retention questions, map each requirement to a mechanism: audit window over deleted data means soft-deleted file retention, and audit of change history means log retention. Set both Delta table properties to the same interval and note that the log default is already 30 days while the file default is only 7 days.
Frequently Asked Questions
Why isn't delta.deletedFileRetentionDuration alone enough to meet the requirement?
Why is delta.enableDeletionVectors wrong here even though deleted rows stay readable?
Discussion 6 contributions
Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.
From public community threads 6
-
Abhi_007 👍 3delta.logRetentionDuration controls how long transaction history/logs are retained (default 30 days). delta.deletedFileRetentionDuration controls how long deleted data files are retained before they become eligible for permanent removal by VACUUM (default 7 days). Setting it to 30 days supports the audit retention requirement.
-
JudgeMaster 👍 1The two properties you should configure are C. delta.logRetentionDuration and D. delta.deletedFileRetentionDuration. Explanation Delta Lake manages its data retention and historical point-in-time querying (time travel) using two interrelated table properties: • delta.deletedFileRetentionDuration: This property determines how long physically deleted data files (from operations like DELETE or MERGE) are kept before they are eligible for permanent removal by a VACUUM operation. Setting this to 30 days satisfies the auditing constraint by ensuring data files are not removed prematurely. • delta.logRetentionDuration: This property dictates how long the table's transaction log (commit history) is kept. To query data as it existed up to 30 days ago for auditing purposes, the log must also be preserved for at least 30 days to re-construct historical states.
-
Raja419 👍 1C. delta.logRetentionDuration - Controls how long the Delta transaction log history is retained. - Default is 30 days. - Supports audit requirements by retaining table history. D. delta.deletedFileRetentionDuration - Controls how long deleted data files are retained before they can be physically removed. - Set it to 30 days. - After 30 days, VACUUM can permanently remove the obsolete data files.
-
zxc01 👍 1enableDeletionVectors to true will mark changes to rows as soft-deletes that logically modify existing Parquet data files in the table. marked files will be kept forever until you execute REORG TABLE. you still can query deleted data by SELECT * FROM my_table VERSION AS OF version number. question said you must confirm order than 30days data has been dropped and minimize admin.
-
zxc01 👍 1It should be B and D. deletedFileRetentionDuration default value is 7 days, you should change it to 30 days. logRetentionDuration default setting is 30 days, you don't need change it. enableDeletionVectors default setting to true which means keep deleted data until you REORG table. you should change it to false.
Show 1 more comment
-
mtc9 👍 1enableDeletionVectors changes it from copy on write to merge on read mode, it doesn't affect retention of deleted files. Although I'm honesty not sure if retaining logs is needed, the question clearly required to retain data, not logs.
Reader comments & corrections
No reader corrections yet — spotted an error or have a note? Share it below.