Enable KFP execution caching to skip unchanged preprocessing
Short answer
A Enable KFP execution caching on the preprocessing component with set_caching_options(enable_caching=True) so unchanged inputs or parameters reuse its cached output and skip the two-hour rerun.
Question
You are currently iterating on the code for the subsequent model_training component and need to re-submit the pipeline frequently for testing. You need to ensure that the preprocessing component only re-runs if its inputs or parameters have changed. You want to minimize effort, costs, and total execution time. What should you do?
Key Takeaways
This question tests KFP execution caching as the built-in input/parameter-aware reuse mechanism; the trap is mistaking artifact persistence or lineage tracking for automatic cache invalidation.
Kubeflow Pipelines execution caching via set_caching_options(enable_caching=True) is the correct answer (A) for reusing the two-hour preprocessing output while iterating on model_training in Agent Platform Pipelines.
Common mistake: The plausible wrong answer is D: a dsl.Condition based on the last hour. Time does not indicate whether preprocessing inputs or parameters changed, so it can reuse stale output or rerun unnecessarily; KFP caching invalidates on the actual execution inputs.
Expert Analysis
Match the qualifier to the mechanism. The question’s decisive phrase is “only re-runs if its inputs or parameters have changed.” That describes KFP execution caching: Agent Platform Pipelines records a component execution by its definition and inputs; when you re-submit the pipeline and those values match a previous execution, the cached output is reused and the two-hour preprocessing step is skipped. Option A exposes exactly that built-in behavior with set_caching_options(enable_caching=True).
A is the minimum-effort solution. You are changing model_training, not preprocessing, and resubmitting frequently. A single component-level caching option reuses the preprocessing output while letting the training component rerun with new code, reducing wall-clock time and compute cost without extra services or custom logic. It also matches the requirement that preprocessing rerun when its inputs or parameters change, because a cache miss causes normal execution.
B is lineage, not automatic reuse. Experiments can record runs and you can pass a prior artifact URI to training, but that is manual artifact management. It does not automatically compare the current preprocessing inputs/parameters and decide to skip; you would build the skip logic and handle artifact compatibility yourself.
C persists data but cannot decide freshness. A shared PVC may hold preprocessed data across runs, but it has no awareness of whether this run’s inputs or parameters differ. You would need custom existence checks, overwrite rules, and concurrency handling, increasing effort and operational risk.
D is time-based, not input-based. A dsl.Condition testing whether preprocessing ran “in the last hour” can reuse stale output after changed inputs or rerun unchanged inputs after an hour. It does not satisfy “only re-runs if its inputs or parameters have changed,” and it is less reliable than KFP caching.
Exam Strategy
When a scenario says a long-running step should rerun only when inputs or parameters differ, scan options for the platform’s native cache control before considering storage or lineage services. In KFP, caching is a component-level setting (set_caching_options), whereas PVCs persist bytes and dsl.Condition gates on runtime logic. Anchor your choice to the exact invalidation condition stated; any solution requiring manual URI plumbing or time windows is a distractor.
Frequently Asked Questions
What makes a KFP execution cache hit for the preprocessing component?
Why is a shared PVC not equivalent to KFP execution caching?
Will caching prevent me from testing changes to model_training?
Community Votes
100% of anonymous learners picked answer A. Votes are pick records left by other test-takers — they are not the verified answer.
Discussion 2 contributions
Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.
From public community threads 2
-
taylz876 👍 1 Answered ARepeated Q280 (and similar to Q201) Why A is right • Caching reuses the output when nothing changed. Agent Platform Pipelines stores a step's output, and if the inputs, parameters and definition are the same on the next run, the step is skipped. The two-hour preprocessing then runs only when its inputs or parameters change, while you re-submit the pipeline to test the training component. • Minimal effort, cost and time: it's one setting. Why the others are wrong • B: Tracking lineage and passing an output URI by hand is manual and doesn't skip the step by itself. • C: A persistent volume shared across runs is custom infrastructure. • D: A condition that checks "ran in the last hour" doesn't check whether the inputs changed.
-
n2183712847 👍 1 Answered ACorrect answer: A. KFP execution caching automatically reuses the preprocessing result when its inputs and parameters are unchanged, avoiding the two-hour recomputation and reducing cost and runtime; B requires manual artifact management, C adds unnecessary persistent-storage complexity, and D uses a time-based condition instead of KFP’s built-in input/parameter-based caching.
Reader comments & corrections
No reader corrections yet — spotted an error or have a note? Share it below.
Related Analysis
Practice All PMLE Questions
Access 65 questions with complete answers and detailed explanations.
View Full PMLE Practice Test →