Managed Service for Apache Airflow for Hybrid Retraining Workflows

PMLE By islever Editorial Last updated

Automating model retraining. Developing end-to-end ML pipelines.

Short answer

C Use Managed Service for Apache Airflow with Airflow operators and sensors to orchestrate the on-premises shell script, Agent Platform Pipelines training, and third-party metrics API.

Question

You are an ML engineer at a retail company. You need to automate a recurring retraining workflow for a demand forecasting model. The workflow involves dependencies across hybrid environments and must execute the following sequential steps:
  1. Trigger a shell script on an on-premises server to extract legacy sales data and send it to a Cloud Storage bucket.
  2. Trigger a pipeline in Agent Platform Pipelines to train and validate the new model.
  3. Push the validation metrics to a third-party API.
You need a fully managed orchestration service that provides native connectors to handle these cross-platform dependencies. You want to use the most stable, robust solution. What should you do?
  1. Use Workflows to define the orchestration logic in YAML. Use the http.post connector to trigger the on-premises shell script and the Agent Platform API for the training job.
  2. Use Cloud Scheduler to trigger a Cloud Run function that executes the on-premises script using SSH, publishes a Pub/Sub message upon completion, and triggers the subsequent workflow steps.
  3. Use Managed Service for Apache Airflow to define a directed acyclic graph (DAG) in Python. Use Apache Airflow operators and sensors to orchestrate the workflow steps. Correct Answer
  4. Use Agent Platform Pipelines to develop a custom container component for each workflow step. Use the Kubeflow Pipelines (KFP) SDK to define the execution order.

Key Takeaways

It tests choosing a fully managed orchestration layer for cross-platform dependencies; the trap is assuming Agent Platform Pipelines or Workflows alone can natively trigger an on-premises shell script and third-party API.

This question tests Managed Service for Apache Airflow as a fully managed orchestrator for hybrid retraining workflows. The correct answer is C because Airflow operators and sensors natively coordinate on-premises scripts, Agent Platform Pipelines, and HTTP API calls.

Common mistake: A common mistake is choosing Agent Platform Pipelines (D) because it is managed and integrates training, but it requires custom container components and lacks native connectors for on-premises shell execution and third-party APIs.

Expert Analysis

Governing requirement The scenario asks for a fully managed orchestration service with native connectors for three heterogeneous steps: an on-premises shell script, an Agent Platform Pipelines training run, and a third-party HTTP API. The deciding principle is separating orchestration from execution: the orchestrator should provide prebuilt integrations and stateful dependency handling while each platform performs its own work. Managed Service for Apache Airflow fits because its operators and sensors are designed for cross-system dependencies, and the service itself is managed.

Why option C wins Managed Service for Apache Airflow lets you define the workflow as a Python DAG. An SSH operator can run the legacy extraction command on the on-premises server, a pipeline operator can launch and monitor the Agent Platform Pipelines training job, and an HTTP operator or sensor can publish validation metrics to the third-party API. Sensors enforce the required sequential order by waiting for each step to complete. This uses documented integrations rather than bespoke glue, making it the most stable and robust choice.

Why the other options fall short Option A relies on Workflows and http.post, which can call HTTP endpoints but do not natively execute a shell script on an on-premises server; you would need to expose and secure an endpoint first. Option B stitches together Cloud Scheduler, Cloud Run, SSH, and Pub/Sub, creating custom infrastructure instead of using a managed orchestration service with native connectors. Option D uses Agent Platform Pipelines with custom container components for every step; it is tightly coupled to ML execution and lacks native handling for the on-premises shell step and third-party API.

Community reasoning Comments on this question align with that reasoning: Airflow offers ready-made operators for remote command execution, pipeline launching, and HTTP API calls, plus sensors to wait for completion. They reject Workflows because its connectors do not directly support the on-premises script, reject the custom SSH/Pub/Sub chain because it adds maintenance, and reject custom Pipelines components as unnecessary for this cross-platform orchestration.

Official Reference

Exam Strategy

When a scenario mixes on-premises commands, Google Cloud pipelines, and external APIs, look for an orchestrator with prebuilt operators and sensors rather than one requiring custom glue. Separate the managed compute for each step from the orchestration layer, and prefer a product whose documented model includes cross-system dependencies. Eliminate options that need custom SSH, Pub/Sub, or container wrappers for every step, because those increase operational burden and violate the fully managed native-connector requirement.

Frequently Asked Questions

Why not use Agent Platform Pipelines for the entire retraining workflow?

Agent Platform Pipelines is suited to ML steps inside Agent Platform, but the on-premises shell script and third-party API would need custom components. The question asks for native cross-platform connectors and a stable managed orchestrator, which Airflow provides.

Does Workflows support triggering an on-premises shell script with http.post?

Workflows can call HTTP endpoints, but an on-premises shell script must be exposed through an accessible service. That is not a native on-premises execution connector, so option A adds architecture not required by the scenario.

Which Airflow features map to the three workflow steps?

An SSH operator can run the legacy extraction command, an Agent Platform operator can launch the training pipeline, and an HTTP operator or sensor can publish validation metrics and wait for completion.

More PMLE FAQ →

Community Votes

C
100%

100% of anonymous learners picked answer C. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 2 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 2

  1. taylz876 👍 1 Answered C
    Why C is right • Hybrid steps in a fixed order: an on-premises script, then a pipeline run, then a call to a third-party API. Airflow (Cloud Composer, now Managed Service for Apache Airflow) has ready-made operators for each kind of step: running a command on a remote server (SSH), launching an Agent Platform pipeline, and calling an HTTP API. Sensors wait for a step to finish before the next one starts. • Fully managed and robust: it's a mature orchestration tool built for cross-platform workflows with retries and monitoring. Why the others are wrong • A: Workflows can call APIs, but triggering an on-premises shell script through an HTTP call needs something on the server to receive it, and it has fewer native connectors for the mix of steps. • B: A function that runs a script over SSH, publishes messages and chains steps by hand is custom plumbing, not a robust orchestrator. • D: Agent Platform Pipelines is for ML steps. It isn't designed to run on-premises scripts or push to third-party services. Note - Managed Service for Apache Airflow: "Airflow orchestrates the data plumbing across many services, but for training and deploying ML models specifically, use Agent Platform Pipelines instead." This workflow spans on-premises and third-party systems, so Airflow is the orchestrator.
  2. n2183712847 👍 1 Answered C
    Correct answer: C. Managed Service for Apache Airflow is a fully managed orchestration platform built for complex workflows with cross-environment dependencies, and its operators/sensors provide robust integration with external systems; A relies on Workflows connectors that may not directly support executing an on-premises shell script, B requires custom SSH/Pub/Sub infrastructure, and D unnecessarily uses ML pipeline components for broader infrastructure orchestration.

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide