CDL — Frequently Asked Questions

BigQuery is the serverless, fully managed data warehouse on Google Cloud that handles petabyte-scale data analytics for strategic decisions. It allows organizations to run SQL queries on massive datasets with sub-second response times, supports built-in machine learning via BigQuery ML, and integrates with visualization tools like Looker Studio. Its architecture separates compute from storage, enabling elastic scaling without manual intervention. Key features include data clustering, partitioning, and the ability to query data directly from Cloud Storage using external tables without loading.

Most organizations exhibit 'cloud spending sprawl' — uncontrolled, decentralized cloud expenditures where teams provision resources without centralized cost oversight. This typically manifests as over-provisioned compute instances, orphaned storage buckets, unused IP addresses, and shadow IT deployments. The typical behavior follows a pattern of rapid initial adoption followed by cost optimization efforts. Best practices include implementing budgets and alerts, using cost allocation tags, regularly auditing resource usage, and adopting FinOps principles to establish accountability and visibility across teams.

Sustained-Use Discounts (SUD) are automatic discounts applied to Compute Engine VM instances that run continuously for more than 25% of a billing month. No upfront commitment or reservation is needed — the discount applies automatically. The discount scales with usage: the longer a VM runs in a given month, the higher the discount, up to 30% for 24/7 usage. If you stop and restart instances within the same month, the sustained-use discount still applies proportionally. SUD cannot be combined with Committed Use Discounts (CUD). For predictable, long-running workloads, CUD offers deeper discounts (up to 55-70%) but requires a 1或3-year commitment.

Redundancy is the design consideration that duplicates critical system components to eliminate single points of failure. In Google Cloud, redundancy is implemented across multiple dimensions: zones (availability zones within a region) and regions (geographic areas). By deploying resources across multiple zones or regions, organizations ensure high availability and disaster recovery. Google Cloud provides global load balancing, multi-region storage classes (like Cross-Region and Multi-Regional), and automatic failover mechanisms. The Shared Responsibility Model means Google handles physical infrastructure redundancy, while customers manage application-level redundancy and data replication strategies.

A good use case for machine learning is predictive maintenance — analyzing sensor data from industrial equipment to predict failures before they occur. Other strong ML use cases include: customer churn prediction (classifying which customers are likely to leave), fraud detection (identifying anomalous transaction patterns), demand forecasting (predicting product demand using historical sales data), and image classification (automating quality control in manufacturing). ML excels when there is sufficient historical data, the pattern is complex for rule-based systems, and the business value of accurate predictions outweighs the cost of building and maintaining the model.

Implementing Error Budgets is a key SRE best practice that improves operational efficiency. An Error Budget is calculated as 1 minus the SLO (e.g., if SLO is 99.9%, the error budget is 0.1%). It quantifies the acceptable amount of failure or downtime, creating a shared metric between development and operations teams. When the error budget is healthy, teams can release features rapidly. When it is depleted, release velocity is throttled to focus on reliability. Other SRE best practices include: defining SLIs (Service Level Indicators) based on user-perceived metrics, using burn rate alerts for proactive incident response, automating remediation actions, and conducting blameless post-mortems to drive continuous improvement.

Google Kubernetes Engine (GKE) is effective for developers resolving application errors because it provides integrated observability tools including Cloud Logging, Cloud Monitoring, and Cloud Trace out of the box. GKE automatically collects structured logs from containers, provides metrics on pod health and resource utilization, and offers distributed tracing for microservices. Developers can use kubectl and the GKE console to inspect pod status, view container logs in real-time, and debug issues without managing the underlying infrastructure. GKE's auto-healing features automatically restart failed containers and replace unhealthy pods, while Horizontal Pod Autoscaler adjusts replica counts based on CPU/memory usage or custom metrics.

During on-premises to cloud migration, the organization is responsible for: (1) Application architecture assessment and refactoring decisions (rehost, refactor, rebuild, replace, or retire); (2) Data migration planning including data mapping, transformation, and validation; (3) Identity and access management configuration (IAM roles, service accounts, permissions); (4) Network architecture design (VPC, subnets, firewall rules, Cloud VPN, Interconnect); (5) Security and compliance configuration (encryption, audit logging, security policies); (6) Cost management and budgeting; (7) Testing and validation of migrated workloads; (8) Training staff on cloud operations. Google is responsible for the underlying infrastructure security (physical data centers, hardware, networking), while the customer secures their data, applications, and access management — this is the Shared Responsibility Model.

Google Cloud protects data from physical intrusion through a multi-layered security approach at its data centers: (1) Perimeter security includes vehicle barriers, perimeter fencing, and 24/7 security guard patrols with rigorous background checks; (2) Biometric access control (eye scanners, fingerprint readers) combined with custom-designed electronic access cards for multi-factor authentication at data center entry points; (3) Laser beam intrusion detection systems throughout the data center floor; (4) 24/7 high-definition CCTV monitoring with interior and exterior cameras; (5) Only less than 1% of Google employees ever set foot in data centers; (6) Custom-designed security chips authenticate servers and peripherals to prevent unauthorized hardware; (7) Strict chain of custody for storage devices from cradle to grave, with multi-step erasure verification before decommissioning. These measures ensure that physical access to infrastructure is extremely restricted and heavily monitored.

Firestore is the recommended NoSQL database for web and mobile applications on Google Cloud. It is a document-based NoSQL database that provides real-time data synchronization, offline support, automatic scaling, and low-latency access. Key features include real-time listeners that automatically update your app when data changes, offline persistence so mobile apps work without connectivity, and native integration with Firebase for rapid mobile app development. For high-throughput operational workloads like IoT time-series data, Cloud Bigtable is more suitable. For complex analytical queries, BigQuery is the better choice. Firestore excels specifically in scenarios requiring real-time sync and offline capabilities for end-user applications.

Yes, Google Cloud customers can transfer data out of Google Cloud at any time. Google Cloud supports data portability as a core principle — customers own their data and can export it using various methods including gsutil (command-line tool), Cloud Storage Transfer Service, direct API calls, or Google Cloud's physical data transfer appliances for large datasets. Standard egress pricing applies for data transferred out to the internet, with free intra-region transfers and reduced pricing for transfers to specific destinations. Google also supports open data formats and standards (like SQL, Apache Avro, Parquet) to ensure applications and data can be moved to other platforms. This addresses vendor lock-in concerns and ensures customers maintain full control over their data lifecycle.

A defining feature of a non-relational (NoSQL) database is its flexible, schema-less data model that does not rely on fixed table structures with predefined rows and columns. Instead, NoSQL databases support various data models including document (Firestore), key-value (Memorystore), wide-column (Bigtable), and graph (Neo4j) formats. Key characteristics include: horizontal scalability across distributed systems, high write and read throughput for specific access patterns, eventual consistency options (though some like Cloud Spanner offer strong consistency), and the ability to store semi-structured or unstructured data. Unlike relational databases, NoSQL databases typically do not support SQL joins or ACID transactions across multiple records, trading relational integrity for scalability and flexibility.

TensorFlow is the premier open-source library for building and training machine learning models on Google Cloud. Developed and maintained by Google, TensorFlow is the most popular ML framework on GitHub and powers many of Google's own services including Search, Gmail, and Maps. Key features include: support for both CPU and GPU/TPU acceleration, TensorFlow Serving for production model deployment, TensorFlow Extended (TFX) for end-to-end ML pipelines, and integration with Kubernetes for distributed training at scale. TensorFlow works seamlessly with Google Cloud services like Vertex AI for managed ML workflows, Cloud TPUs for accelerated training, and BigQuery ML for SQL-based machine learning. Keras is now integrated as TensorFlow's high-level API for easier model building.

To mitigate cloud vendor lock-in with open standards, organizations should: (1) Use containerization (Docker) and orchestration (Kubernetes/GKE) to package applications in a cloud-agnostic format that can run on any cloud provider; (2) Adopt Infrastructure as Code (IaC) tools like Terraform or OpenTofu to define cloud resources in declarative configuration files that can target multiple providers; (3) Prefer open-source technologies and standards-based APIs over proprietary services; (4) Design applications with loosely coupled microservices architecture to isolate dependencies; (5) Use standard data formats (SQL, Apache Parquet, Avro) for data portability; (6) Implement an abstraction layer or API gateway to decouple application logic from specific cloud services; (7) Evaluate multi-cloud strategies using platforms like Google Anthos that support hybrid and multi-cloud deployments; (8) Regularly audit dependencies on proprietary services and plan migration paths.

The Service Level Indicator (SLI) is the SRE concept that measures how well a system is performing from the user's perspective. An SLI is a quantitative measure of service quality, such as request latency, error rate, throughput, or availability (e.g., percentage of successful HTTP 200 responses). SLIs are measured using various data sources including server-side logs, client-side metrics, infrastructure monitoring, and distributed tracing (OpenTelemetry). The SLI feeds into the Service Level Objective (SLO), which is the target threshold (e.g., '99.9% of requests complete within 300ms'). The Error Budget (1 - SLO) then determines how much risk the team can take. Google recommends keeping SLIs simple, user-focused, and limited to 3-5 per user journey to avoid alert fatigue and measurement overhead.

Google Cloud's resource hierarchy (Organization > Folders > Projects > Resources) enables centralized security policy implementation through inheritance. IAM policies applied at a higher level automatically propagate downward — a policy set on an Organization node applies to all folders, projects, and resources beneath it. This allows administrators to enforce security standards consistently across the enterprise. For example, a 'deny' policy blocking external IP addresses at the Organization level prevents any project from creating publicly accessible instances. Conversely, more permissive access can be granted at lower levels (e.g., Editor role at project level even if Viewer at organization level). Deny policies always override allow policies. This hierarchical approach enables the principle of least privilege while maintaining operational flexibility — teams get default-restricted access but can request specific permissions through their project or folder administrators.

Cloud Run is the Google Cloud service that hosts containers serverlessly with automatic scaling. It allows developers to deploy any containerized application without managing servers or clusters. Cloud Run automatically scales from zero to thousands of requests based on traffic, handling all infrastructure provisioning, load balancing, and scaling decisions. Key features include: request-based autoscaling (scales to zero when no requests arrive), support for HTTP/2 and gRPC, built-in SSL certificates, and integration with Cloud Build for CI/CD pipelines. It supports both stateless containers and can connect to managed databases like Cloud SQL and Cloud Spanner. For container orchestration with more control over the cluster, Google Kubernetes Engine (GKE) is the alternative, but Cloud Run is preferred for simple container deployment without cluster management overhead.

Natural Language Processing (NLP) and Computer Vision are the primary ML scenarios that unlock value from unstructured data. For text data, Google Cloud's Natural Language API and AutoML Natural Language can perform sentiment analysis, entity recognition, syntax analysis, text classification, and document extraction — turning unstructured text (emails, reviews, documents) into structured, actionable insights. For image data, AutoML Vision and the Cloud Vision API can classify images, detect objects, extract text (OCR), and analyze visual content. Google's AutoML Translation handles specialized domain translations. These pre-trained and customizable models allow organizations to process the over 90% of enterprise data that is unstructured (documents, emails, scanned PDFs, images) without needing deep ML expertise, converting 'dark data' into searchable, analyzable information.

Google Cloud's Data Loss Prevention (DLP) API is the quickest option for PII detection in photos. The DLP API can scan images for personally identifiable information such as names, phone numbers, email addresses, and other sensitive data using built-in infoTypes (sensitive data types). It supports both structured and unstructured data including images, text files, and BigQuery tables. For photos specifically, the DLP API uses OCR (Optical Character Recognition) to extract text from images and then scans for PII patterns. Integration is as simple as calling the DLP inspect endpoint with the image file — no model training required. For more advanced or custom PII detection, organizations can use DLP with AutoML to train custom models, but for quickest time-to-value, the pre-built DLP API with its extensive library of built-in detectors is the recommended starting point.

Platform as a Service (PaaS) is the cloud service model that fits a custom application without infrastructure management. With PaaS, the cloud provider (Google Cloud) manages all underlying infrastructure including servers, operating systems, patching, and scaling, while the customer focuses solely on writing and deploying application code. Google App Engine is Google's PaaS offering — developers upload code in supported languages (Python, Java, Go, .NET, PHP, Node.js) or custom containers, and App Engine automatically handles deployment, load balancing, and auto-scaling. Other PaaS options on Google Cloud include Cloud Run (for containerized apps) and Cloud Functions (for event-driven serverless functions). PaaS sits between IaaS (where you manage OS and above) and SaaS (where you only use the software), offering the right balance of flexibility and managed convenience for custom application development.