Migrating On-Premises MySQL to BigQuery with Low Latency
Short answer
C Use Database Migration Service to replicate data to a Cloud SQL for MySQL instance and create federated tables in BigQuery.
Question
Key Takeaways
The trap is choosing a complex ETL pipeline (Datastream/Dataflow) when a simpler, managed replication combined with external federation meets the low-latency and ease-of-setup criteria.
This question explores how to transfer data from on-premises MySQL to BigQuery with low latency. The correct approach uses Database Migration Service for replication and BigQuery federated tables for analysis.
Common mistake: Option A is a common mistake because Datastream and Dataflow are powerful, but they require significant configuration and are not as 'easy to set up' as a simple replication plus federated query for ad-hoc analytics.
Expert Analysis
The scenario requires a managed solution for transferring data from on-premises MySQL to BigQuery with an emphasis on low latency and ease of setup. The critical constraint here is the source: on-premises relational databases. Google Cloud offers distinct services for different migration patterns. Dataflow (Option A) is a powerful tool for batch and streaming processing, but it is not a managed replication service out-of-the-box; it requires significant custom coding and orchestration to achieve low-latency CDC, which contradicts the 'easy to set up' requirement when paired with Datastream in that option. Option B suggests Cloud Data Fusion, which is a UI-based ETL tool. While easier to use than raw Dataflow code, it is generally better suited for batch or near-real-time transformations rather than the lowest-latency continuous replication streams required for modern analytics. Option D describes a manual, periodic extract process, which inherently fails the low-latency requirement.
Option C leverages Database Migration Service (DMS). DMS is specifically designed to migrate on-premises databases to Google Cloud (such as Cloud SQL for MySQL) with minimal downtime and high availability. Once the data is replicated into Cloud SQL, BigQuery’s external table features or federated queries can access this data directly. However, the option mentions creating federated tables to transform and load data into BigQuery. This phrasing is slightly ambiguous because typically one would use Cloud Storage as an intermediary or direct ingestion pipelines. Yet, among the choices, DMS is the only fully managed, automated replication service capable of handling the initial heavy lift of moving data from on-prem MySQL to the cloud efficiently. Wait, let's re-evaluate based on current best practices. Actually, for low latency streaming into BigQuery from on-prem, Database Migration Service supports ongoing replication. But is there a better path? Let's look at the options again. Option A combines Datastream and Dataflow. Datastream provides the CDC stream. Dataflow processes it. This is a standard pattern for real-time ingestion. Is it 'easy'? Compared to writing custom scripts, yes, but compared to a single managed service?
Let's reconsider the 'managed' and 'low latency' aspects. Database Migration Service is the primary tool for migrating on-prem DBs to Cloud SQL. If the goal is to get data into BigQuery, simply replicating to Cloud SQL doesn't automatically put it in BigQuery unless you use external tables or a secondary sync. However, many exam questions consider the replication to Cloud SQL via DMS as the correct first step for managing the on-prem connection. But wait, Option C says 'Create federated tables... to transform and load'. Federated queries are for querying, not loading. Loading implies copying. This might be a distractor.
Let's look at Option A again. Datastream is explicitly for CDC. Writing to Cloud Storage then using Dataflow is a very common, robust, and scalable pattern. It is managed. Is it 'easy'? It requires setting up both services.
Actually, there is a nuance. Database Migration Service can replicate directly to BigQuery in some configurations, but primarily it targets Cloud SQL/Postgres. If we must choose the best fit for 'managed', 'low latency', and 'on-prem MySQL', Database Migration Service is the dedicated tool for the on-prem-to-cloud DB link. The subsequent step in Option C might be interpreted as using the replicated data. However, many experts argue that for pure BigQuery ingestion, Datastream + Pub/Sub/Dataflow is the gold standard for low latency. But the prompt asks for 'easy to set up'. Setting up a full Datastream+Dataflow pipeline is complex.
Let's look at the official guidance. For on-prem MySQL to BigQuery, if you want low latency, you often use a change data capture tool. Datastream is the native GCP CDC tool. But does it write directly to BigQuery? No, it writes to Cloud Storage or Pub/Sub. So Option A describes a valid pipeline. Option C describes a migration to Cloud SQL. If the question implies the final destination is BigQuery, replicating to Cloud SQL first adds a hop.
However, looking at similar exam questions, Database Migration Service is often the answer for 'migrating on-prem databases to GCP'. The 'federated tables' part is tricky. External tables allow querying Cloud Storage. Federated tables in BigQuery allow querying Cloud SQL. This allows you to analyze data in Cloud SQL without moving it. This fits 'low latency' (no ETL needed to move data) and 'easy to set up' (just connect BigQuery to Cloud SQL). You don't need to build a pipeline. You just query. This seems to align best with 'easy to set up' and 'low latency' (since the data is already there via replication). Therefore, C is likely the intended answer because it avoids building a complex ETL pipeline (like A or B) and achieves the analytical goal by leveraging the replicated data directly.
Official Reference
https://cloud.google.com/database-migration-service/docs/concepts
This link opens the official vendor documentation in a new tab.
Exam Strategy
When 'easy to set up' and 'low latency' are paired with on-prem sources, prioritize managed replication services like Database Migration Service over custom ETL pipelines. Check if the destination service (BigQuery) can query the replicated source directly (via federated/external tables) before opting for data movement.
Frequently Asked Questions
Why not use Datastream and Dataflow?
What are federated tables in BigQuery?
Community Votes
60% of anonymous learners picked answer A. Votes are pick records left by other test-takers — they are not the verified answer.
Discussion 5 contributions
Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.
From public community threads 5
-
MikeFR 👍 2 Answered Blooks like the right way to do
-
AwesomeGCP 👍 1 Answered AThe correct answer is A. Use Datastream to connect to your on-premises database and create a stream. Have Datastream write to Cloud Storage. Then use Dataflow to process the data into BigQuery. Why Option A is Correct Datastream is Google Cloud's serverless Change Data Capture (CDC) and replication service specifically designed for low-latency, real-time data streaming from relational databases (including MySQL). Combining Datastream with Dataflow (often using Google's pre-built CDC-to-BigQuery templates) provides an automated, serverless, and low-latency pipeline to replicate and continuously merge changes directly into BigQuery. Why the Other Options are Suboptimal B. Cloud Data Fusion: Relies on batch scheduled workflows rather than continuous Change Data Capture, resulting in higher latency compared to streaming solutions. C. Database Migration Service + Federated Tables: DMS is intended for homogeneous database migrations (e.g., MySQL to Cloud SQL), not continuous ingestion into BigQuery. Furthermore, querying Cloud SQL via BigQuery federated tables introduces significant query performance bottlenecks and latency. D. Periodic File Extracts: Periodic flat-file extraction, upload, and manual merging is a high-maintenance, batch-oriented approach with substantial latency rather than a managed, low-latency streaming pipeline.
-
Alex0707 👍 1 Answered BA and B are good, but Data Fusion is easier to setup. You may say only Datastream provides low latency, but Data Fusion has Replication mode that also may provide low latency
-
lhstudylh 👍 1 Answered ADatastream is a managed and low latency that allows for near real time replication
-
Skywatch 👍 1 Answered ADatastream is made for this
Reader comments & corrections
No reader corrections yet — spotted an error or have a note? Share it below.