Preprocessing BigQuery Data for TensorFlow Fraud Detection Models
Short answer
C Perform preprocessing in BigQuery using SQL and read data directly into TensorFlow with BigQueryClient for cost-effective, efficient training tied to BigQuery batch inference.
Question
Key Takeaways
This question tests whether you keep preprocessing in the same serverless warehouse where the data already resides and where inference will run, rather than introducing Spark, Dataflow, or pandas.
For large transaction data already in BigQuery that will be used for batch inference in BigQuery, the most cost-effective and efficient preprocessing approach is to transform the data with SQL in BigQuery and read it directly into TensorFlow using BigQueryClient (C).
Common mistake: Choosing D (Apache Beam on Dataflow) is tempting because it is scalable, but it adds pipeline complexity, runner cost, and CSV materialization in Cloud Storage—unnecessary when the data is already in BigQuery.
Expert Analysis
Data locality drives the answer
The transactions already live in a large BigQuery table, and the trained model will later run batch inference in BigQuery. Keeping the preprocessing inside BigQuery avoids exporting data, standing up clusters, or materializing intermediate files. Option C performs preprocessing with SQL directly in BigQuery and then streams the result into TensorFlow via the BigQueryClient, so the data never leaves the warehouse until training reads it.
Why the alternatives fail the constraints
- A (Spark on Dataproc) and D (Beam on Dataflow) both require provisioning a separate processing engine and writing the preprocessed output as CSV files in Cloud Storage. That adds cluster or runner cost, operational complexity, and extra storage—contradicting "cost-effective and efficient." One community comment notes that while these are scalable, they "introduce extra complexity for managing" pipelines and data movement.
- B (pandas DataFrame) forces the large table into a single machine's memory, which will cause resource constraints and performance issues on a large dataset. It also breaks the BigQuery-centric inference path.
The request path for option C
- Write SQL transforms (filtering, joins, feature engineering) as a BigQuery query or view.
- In TensorFlow, use the
BigQueryClient(fromtensorflow.python.framework.bigquery) to read rows directly from that query/table in streaming batches. - Train the Keras model on the streamed tensors.
- Export the trained model and register it for BigQuery batch inference (e.g., via BigQuery ML or a remote model), so inference reuses the same SQL feature logic.
This path minimizes cost (serverless, per-byte pricing), maximizes efficiency (BigQuery's distributed execution), and keeps preprocessing and inference in one consistent SQL environment. The community comments agree, with one summarizing: "Easiest to preprocess the data on BigQuery," and another explaining that the whole workflow should stay in BigQuery because it is "serverless, fast on large tables and cost-effective."
Official Reference
- https://cloud.google.com/bigquery/docs/bigqueryml
- https://www.tensorflow.org/api_docs/python/tf/bigquery
These links open the official vendor documentation in a new tab.
Exam Strategy
When a question states data is already in BigQuery and inference will also happen in BigQuery, look for the option that keeps the entire workflow inside BigQuery. Eliminate any choice that exports data to Cloud Storage or spins up a separate cluster unless the scenario explicitly demands distributed processing outside the warehouse. Also rule out pandas for large tables because of memory limits.
Community Votes
88% of anonymous learners picked answer C. Votes are pick records left by other test-takers — they are not the verified answer.
Discussion 4 contributions
Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.
From public community threads 4
-
pikachu007 👍 5 Answered CA. Spark on Dataproc: While powerful, it incurs additional cluster setup and management costs, potentially less cost-effective for this specific use case. B. pandas DataFrame: Loading large datasets into memory might lead to resource constraints and performance issues, especially for large-scale preprocessing. D. Apache Beam on Dataflow: While scalable, it introduces extra complexity for managing a separate pipeline and storage for preprocessed data.
-
pinimichele01 👍 2Easiest to preprocess the data on BigQuery.
-
pinimichele01 👍 2 Answered Cwent with C
-
taylz876 👍 1 Answered CWhy C is right The transactions are already in a large BigQuery table, and the trained model will run batch inference in BigQuery. So the whole workflow should stay in BigQuery: • Preprocess with SQL in BigQuery: it's serverless, fast on large tables and cost-effective. There is no cluster, no pipeline and no data movement. • Read the data straight into training: TensorFlow's BigQuery reader streams the rows directly into the training pipeline, so you don't need intermediate files. • Consistency for batch inference: since inference also happens in BigQuery, the same SQL can preprocess the data for scoring. Why the others are wrong • A: Spark on Dataproc (now Managed Service for Apache Spark) means running a cluster and exporting data out of BigQuery to CSV files. It's slower and costs more. • B: Loading a large table into a pandas DataFrame needs a lot of memory, and it won't scale to a table this size. • D: Beam on Dataflow works, but it's a heavier pipeline than a SQL statement. It also saves to CSV files, which adds an extra copy of the data and another step.
Reader comments & corrections
No reader corrections yet — spotted an error or have a note? Share it below.
Related Analysis
Practice All PMLE Questions
Access 65 questions with complete answers and detailed explanations.
View Full PMLE Practice Test →