How to Improve Data Discoverability Across BigQuery, Pub/Sub, and PostgreSQL?

PDE By islever Editorial Last updated

Short answer

C Use Data Catalog to automatically catalog BigQuery datasets and Pub/Sub topics. Use custom connectors to manually catalog PostgreSQL tables.

Question

Your organization's data assets are stored in BigQuery, Pub/Sub, and a PostgreSQL instance running on Compute Engine. Because there are multiple domains and diverse teams using the data, teams in your organization are unable to discover existing data assets. You need to design a solution to improve data discoverability while keeping development and configuration efforts to a minimum. What should you do?
  1. Use Data Catalog to automatically catalog BigQuery datasets. Use Data Catalog APIs to manually catalog Pub/Sub topics and PostgreSQL tables.
  2. Use Data Catalog to automatically catalog BigQuery datasets and Pub/Sub topics. Use Data Catalog APIs to manually catalog PostgreSQL tables.
  3. Use Data Catalog to automatically catalog BigQuery datasets and Pub/Sub topics. Use custom connectors to manually catalog PostgreSQL tables. Correct Answer
  4. Use customer connectors to manually catalog BigQuery datasets, Pub/Sub topics, and PostgreSQL tables.

Key Takeaways

This question tests your understanding of Data Catalog's automatic ingestion for Google Cloud sources and supported community connectors for unsupported sources; the trap is incorrectly choosing API custom entries when a lower-effort connector exists.

To improve data discoverability across BigQuery, Pub/Sub, and PostgreSQL, use Data Catalog's native integrations for Google Cloud sources and a community connector for PostgreSQL. The community favors option C, which uses the PostgreSQL connector instead of manually coding API entry creation.

Common mistake: Option B is the most common wrong choice because it correctly uses Data Catalog for BigQuery and Pub/Sub but forces PostgreSQL to be manually integrated via APIs, ignoring that a community connector for PostgreSQL is already available and minimizes effort.

Expert Analysis

Why the Answer Is Correct

Data Catalog automatically catalogs BigQuery datasets and Pub/Sub topics as managed Google Cloud sources. For PostgreSQL, which is not natively ingested, Google's documentation recommends using community-contributed connectors when one is available, and a PostgreSQL connector exists. Option C follows this guidance, keeping development and configuration effort minimal.

Why the Other Options Are Wrong

Option A fails to use Data Catalog's automatic Pub/Sub support and incorrectly handles both Pub/Sub and PostgreSQL via APIs, adding unnecessary effort. Option B is close but still uses APIs for PostgreSQL instead of the lower-effort community connector. Option D ignores Data Catalog's native integrations for BigQuery and Pub/Sub, requiring custom connectors for everything and increasing complexity.

Community Comment Notes

Commenters supporting B often cite the Data Catalog API for external databases, but supporters of C point out that Google's own documentation recommends using a connector when available and only falling back to the API when no connector exists (comments 2, 3, 4, 6). Comment 7 raises a nuance about connectors being for on-premises sources, but this does not block using the PostgreSQL connector for a Compute Engine instance. Overall, the consensus and official guidance favor C.

Official Reference

These links open the official vendor documentation in a new tab.

Exam Strategy

When choosing between API and connector options, prioritize options that use supported community connectors for non-native sources like PostgreSQL; this aligns with Google's recommended path and satisfies 'minimum development effort' requirements.

Community Votes

C
52%
B
48%

52% of anonymous learners picked answer C. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 21 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 21

  1. raaad 👍 13 Answered B
    - It utilizes Data Catalog's native support for both BigQuery datasets and Pub/Sub topics. - For PostgreSQL tables running on a Compute Engine instance, you'd use Data Catalog APIs to create custom entries, as Data Catalog does not automatically discover external databases like PostgreSQL.
  2. datapassionate 👍 12 Answered C
    Data Catalog is the best choice. But for catalogging PostgreSQL it is better to use a connector when available, instead of using API. https://cloud.google.com/data-catalog/docs/integrate-data-sources#integrate_unsupported_data_sources
  3. ML6 👍 5 Answered C
    Google Recommendation: If you can't find a connector for your data source, you can still manually integrate it by creating entry groups and custom entries. To do that, you can: - Use one of the Data Catalog Client Libraries in one of the following languages: C#, Go, Java, Node.js, PHP, Python, or Ruby. - Or manually build on the Data Catalog API. However, there is a connector for PostgreSQL, so option C.
  4. baimus 👍 3 Answered C
    This is C. To clarify some issues below with B, the links provided by supporters of B actually do say that it's preferable to use a community connector where available, and to only use the API when the case is genuinely not supported by community connectors. In this case it's Postgresql, so it's supported, see here for full list: https://cloud.google.com/data-catalog/docs/integrate-data-sources#integrate_on-premises_data_sources So this would be B if it was something like Q+ or some genuinely unsupported database, but postgres is supported for community connector.
  5. fitri001 👍 3 Answered B
    BigQuery Datasets and Pub/Sub Topics: Google Data Catalog can automatically catalog metadata from BigQuery and Pub/Sub, making it easy to discover and manage these data assets without additional development effort. PostgreSQL Tables: While Data Catalog does not have built-in connectors for PostgreSQL, you can use the Data Catalog APIs to manually catalog the PostgreSQL tables. This requires some custom development but is manageable compared to creating custom connectors for everything.
Show 16 more comments
  1. joao_01 👍 3
    In the opction C, the expression "Use custom connectors to manually catalog PostgreSQL tables." is refering to the use case of Google when you want to use "Community-contributed connectors to multiple popular on-premises data sources". As you can see, this connectors are for ON-PREMISSES data sources ONLY. In this case the Postgres is in a VM in the cloud. Thus, the option correct is B.
  2. LaxmanTiwari 👍 3 Answered C
    I vote for c as per Integrate on-premises data sources To integrate on-premises data sources, you can use the corresponding Python connectors contributed by the community: under the link https://cloud.google.com/data-catalog/docs/integrate-data-sources
  3. saschak94 👍 3 Answered C
    If you can't find a connector for your data source, you can still manually integrate it by creating entry groups and custom entries. To do that, you can: - Manually build on the Data Catalog API.
  4. Matt_108 👍 3 Answered B
    Option B - Data Catalog automatically maps out GCP resources and dev efforts are minimized by leveraging the data catalog API to do the same for postgresql db
  5. shanks_t 👍 2 Answered B
    Data Catalog automatically catalogs metadata from Google Cloud sources such as BigQuery, Vertex AI, Pub/Sub, Spanner, Bigtable, and more. To catalog metadata from non-Google Cloud systems in your organization, you can use the following: Community-contributed connectors to multiple popular on-premises data sources Manually build on the Data Catalog APIs for custom entries
  6. GCP001 👍 2
    B. -- Looks much better option as needed low development efforts. -- C not looking right as it will need lot of dev efforts for custom connectors.
  7. 987af6b 👍 2 Answered C
    I’m voting for C because the documentation states that Postgres is a custom connector developed by the community.
  8. Abizi 👍 1 Answered C
    Why C is Correct? BigQuery datasets → ✅ Automatically cataloged in Data Catalog Pub/Sub topics → ✅ Automatically cataloged in Data Catalog PostgreSQL on Compute Engine → ❌ Not automatically cataloged Requires a custom connector to extract metadata and push it to Data Catalog. Option B (using Data Catalog APIs manually) is not enough because PostgreSQL metadata isn’t natively supported. Why Not B? Option B suggests using Data Catalog APIs manually for PostgreSQL. However, Data Catalog does not natively support PostgreSQL metadata extraction. You need a custom connector to first extract PostgreSQL schema information, then push it to Data Catalog.
  9. Cassim 👍 1 Answered B
    Option B leverages Data Catalog to automatically catalog BigQuery datasets and Pub/Sub topics, which streamlines the process and reduces manual effort. Using Data Catalog APIs to manually catalog PostgreSQL tables ensures consistency across all data assets while minimizing development and configuration efforts.
  10. hanoverquay 👍 1 Answered B
    option B, there's no need to build a custom connector now, postgreSQL is now supported https://github.com/GoogleCloudPlatform/datacatalog-connectors-rdbms/tree/master/google-datacatalog-postgresql-connector
  11. Pime13 👍 1 Answered A
    https://cloud.google.com/data-catalog/docs/concepts/overview#automatic_cataloging_of_assets https://cloud.google.com/data-catalog/docs/concepts/overview#catalog-non-google-cloud-assets
  12. virat_kohli 👍 1 Answered B
    B. Use Data Catalog to automatically catalog BigQuery datasets and Pub/Sub topics. Use Data Catalog APIs to manually catalog PostgreSQL tables.
  13. Y___ash 👍 1 Answered B
    Use Data Catalog to automatically catalog BigQuery datasets and Pub/Sub topics. Use Data Catalog APIs to manually catalog PostgreSQL tables.
  14. AWSandeep 👍 1 Answered B
    This section explains it clearly: https://cloud.google.com/data-catalog/docs/integrate-data-sources#integrate_unsupported_data_sources.
  15. Harshzh12 👍 1 Answered B
    Datacatalog API contain the connector for postgresql with using it developer don't have to create the custom connectors
  16. meh_33 👍 1
    raaad mostly correct and we can check his description supporting his answer so we can go with it .Cheers mate

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

How to establish consistent BigQuery spend?

S-grade

Triggering Airflow DAGs Reactively in Private Cloud Composer

S-grade

How to resolve Cloud Composer 2 worker pod evictions and task failures?

S-grade

Which Google Cloud database for on-prem PostgreSQL with analytics?

S-grade

← Back to PDE Study Guide