Storing time-series data for low latency lookups and analytics?

PDE By islever Editorial Last updated

Short answer

B Store your data in Bigtable. Concatenate the sensor ID and timestamp and use it as the row key. Perform an export to BigQuery every day.

Question

You have a network of 1000 sensors. The sensors generate time series data: one metric per sensor per second, along with a timestamp. You already have 1 TB of data, and expect the data to grow by 1 GB every day. You need to access this data in two ways. The first access pattern requires retrieving the metric from one specific sensor stored at a specific timestamp, with a median single-digit millisecond latency. The second access pattern requires running complex analytic queries on the data, including joins, once a day. How should you store this data?
  1. Store your data in BigQuery. Concatenate the sensor ID and timestamp, and use it as the primary key.
  2. Store your data in Bigtable. Concatenate the sensor ID and timestamp and use it as the row key. Perform an export to BigQuery every day. Correct Answer
  3. Store your data in Bigtable. Concatenate the sensor ID and metric, and use it as the row key. Perform an export to BigQuery every day.
  4. Store your data in BigQuery. Use the metric as a primary key.

Key Takeaways

This tests selecting the right database for specific access patterns; the trap is choosing BigQuery for low-latency lookups or designing a poor Bigtable row key that causes hotspotting.

Use Bigtable for high-throughput, low-latency time-series data and export to BigQuery for analytics. The community confirms this hybrid approach effectively meets the strict latency and complex query requirements.

Common mistake: Choosing Option A because BigQuery handles analytics, but ignoring the single-digit millisecond latency requirement for point lookups.

Expert Analysis

Why the Answer Is Correct

Bigtable is designed for high throughput and low latency point reads, easily achieving single-digit millisecond latency. Constructing the row key by concatenating the Sensor ID and timestamp ensures efficient retrieval of specific data points and prevents hotspotting. The daily export to BigQuery satisfies the requirement for complex analytical queries involving joins, leveraging BigQuery's OLAP strengths.

Why the Other Options Are Wrong

Options A and D use BigQuery, which is an OLAP data warehouse not optimized for the low-latency point lookups required (it typically takes seconds to return results). Option C suggests a poor row key design (Sensor ID + metric), which would cause hotspotting because all time-series data for a single sensor would be written to the same node, and it fails to allow efficient retrieval by timestamp.

Community Comment Notes

The community strongly supports Option B, with top-rated comments highlighting Bigtable's speed and scalability. Users specifically note that the wide-column design is ideal for time-series data and that the proposed row key structure is the standard best practice for this type of workload.

Official Reference

These links open the official vendor documentation in a new tab.

Exam Strategy

Identify the specific latency requirements. If you see "single-digit millisecond latency" for lookups, immediately look for a NoSQL option like Bigtable rather than a data warehouse like BigQuery.

Community Votes

B
100%

100% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 6 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 6

  1. raaad 👍 16 Answered B
    - Bigtable excels at incredibly fast lookups by row key, often reaching single-digit millisecond latencies. - Constructing the row key with sensor ID and timestamp enables efficient retrieval of specific sensor readings at exact timestamps. - Bigtable's wide-column design effectively stores time series data, allowing for flexible addition of new metrics without schema changes. - Bigtable scales horizontally to accommodate massive datasets (petabytes or more), easily handling the expected data growth.
  2. scaenruy 👍 3 Answered B
    B. Store your data in Bigtable. Concatenate the sensor ID and timestamp and use it as the row key. Perform an export to BigQuery every day.
  3. fitri001 👍 2 Answered B
    agree with raaad
  4. Matt_108 👍 1 Answered B
    Option B - agree with raaad
  5. JyoGCP 👍 1 Answered B
    Option B
Show 1 more comment
  1. hanoverquay 👍 1 Answered B
    voted b

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

How to establish consistent BigQuery spend?

S-grade

Triggering Airflow DAGs Reactively in Private Cloud Composer

S-grade

How to resolve Cloud Composer 2 worker pod evictions and task failures?

S-grade

Which Google Cloud database for on-prem PostgreSQL with analytics?

S-grade

← Back to PDE Study Guide