How to configure Dataflow hopping windows for real-time aggregation?

You are designing a real-time system for a ride hailing app that identifies areas with high demand for rides to effectively reroute available drivers to meet the demand. The system ingests data from multiple sources to Pub/Sub, processes the data, and stores the results for visualization and analysis in real-time dashboards. The data sources include driver location updates every 5 seconds and app-based booking events from riders. The data processing involves real-time aggregation of supply and demand data for the last 30 seconds, every 2 seconds, and storing the results in a low-latency system for visualization. What should you do?

  1. Group the data by using a tumbling window in a Dataflow pipeline, and write the aggregated data to Memorystore.
  2. Group the data by using a hopping window in a Dataflow pipeline, and write the aggregated data to Memorystore. Source Reference Answer
  3. Group the data by using a session window in a Dataflow pipeline, and write the aggregated data to BigQuery.
  4. Group the data by using a hopping window in a Dataflow pipeline, and write the aggregated data to BigQuery.

Community Votes

B
100%

100% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Community Insight

The requirement to view a rolling time period (last 30s) at a high frequency (every 2s) defines a hopping window, while 'low-latency' dictates the use of an in-memory store like Memorystore.

The correct approach uses a Dataflow hopping window to aggregate data over the last 30 seconds every 2 seconds, storing results in Memorystore for low-latency access.

Selecting a tumbling window (Option A) because it is simpler, not realizing that non-overlapping tumbling windows cannot maintain the context of the last 30 seconds when updating every 2 seconds.

Community Discussion (7 comments)

raaad 👍 12 Selected: B
  • Hopping Window: Hopping windows are fixed-sized, overlapping intervals. - Aggregate data over the last 30 seconds, every 2 seconds, as hopping windows allow for overlapping data analysis. - Memorystore: Ideal for low-latency access required for real-time visualization and analysis.
Jeyaraj 👍 1
OPTION A. (IGNORE MY Previous Comment) Tumbling windows are the best choice for this ride-hailing app because they provide accurate 2-second aggregations without the complexities of overlapping data. This is crucial for real-time decision-making and ensuring accurate visualization of supply and demand. Hopping windows introduce potential inaccuracies and complexity, making them less suitable for this scenario. While they can be useful in other situations, they are not the optimal choice for real-time aggregation with strict accuracy requirements.
Jeyaraj 👍 1
Option B. Tumbling windows are the best choice for this ride-hailing app because they provide accurate 2-second aggregations without the complexities of overlapping data. This is crucial for real-time decision-making and ensuring accurate visualization of supply and demand. Hopping windows introduce potential inaccuracies and complexity, making them less suitable for this scenario. While they can be useful in other situations, they are not the optimal choice for real-time aggregation with strict accuracy requirements.
JyoGCP 👍 1 Selected: B
Option B
ashdam 👍 1
hopping window is clear but memorystore vs bigquery?? Why memorystore and not bigquery?
Jordan18 👍 1
why not D?
scaenruy 👍 2 Selected: B
B. Group the data by using a hopping window in a Dataflow pipeline, and write the aggregated data to Memorystore.

Comments & Corrections

No comments yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Submitted for moderation before publishing. Keep it helpful and respectful.

Expert Analysis

Why the Answer Is Correct

Option B is correct because a hopping window with a size of 30 seconds and a hop (interval) of 2 seconds exactly fulfills the requirement to aggregate supply and demand for the 'last 30 seconds, every 2 seconds.' Memorystore is chosen because the question specifies a 'low-latency system for visualization,' and in-memory stores like Redis (Memorystore) provide the necessary speed for real-time dashboards.

Why the Other Options Are Wrong

Option A is incorrect because tumbling windows represent discrete, non-overlapping time buckets; a 2-second tumbling window cannot aggregate data from the previous 28 seconds. Option C is incorrect because session windows group data based on periods of activity separated by gaps, not fixed time intervals. Option D uses the correct windowing strategy but BigQuery is an analytical warehouse better suited for batch or historical analysis, not the sub-second latency required for real-time operational dashboards.

Community Comment Notes

The community overwhelmingly supports Option B, highlighting that hopping windows are designed for overlapping analysis, which is critical for calculating rolling metrics. Comments also clarify that while BigQuery is powerful, the specific requirement for 'low-latency' visualization makes Memorystore the superior storage choice in this context.

Official Reference

Exam Strategy

When you see a requirement to aggregate 'over X time, every Y time' (where Y < X), immediately identify a hopping window. Look for keywords like 'low-latency' or 'real-time dashboard' to select in-memory storage (Memorystore/Redis) over BigQuery.

Related Analysis

You have thousands of Apache Spark jobs running in your on-p

S-Grade

You are on the data governance team and are implementing sec

S-Grade

You are preparing an organization-wide dataset. You need to

S-Grade

← Back to PDE Study Guide