2 packs: 15% off with code BUY2. 3 or more: 20% off with code BUY3.

Free Google Professional Data Engineer Practice Questions

87 free Google Professional Data Engineer practice questions from the CertStash pack of 349, each with the correct answer and a full explanation. The rule is the same for every exam: ten free on the product page, 87 free here, all 349 in the pack, and each step includes the one before it.

Question 1

Your company is in a highly regulated industry. One of your requirements is to ensure individual users have access only to the minimum amount of information required to do their jobs. You want to enforce this requirement with Google BigQuery. Which three approaches can you take? (Choose three.)

  1. Disable writes to certain tables.
  2. Restrict access to tables by role.
  3. Ensure that the data is encrypted at all times.
  4. Restrict BigQuery API access to approved users.
  5. Segregate data across multiple tables or databases.
  6. Use Google Stackdriver Audit Logging to determine policy violations.
Show answer and explanation

Correct answer: B, D, E. Restrict access to tables by role; Restrict BigQuery API access to approved users; Segregate data across multiple tables or databases.

To enforce minimum access requirements, you should restrict access to tables by role (B), allowing different users different permissions based on job function. Restricting BigQuery API access to approved users (D) ensures only authorized personnel can query data. Segregating data across multiple tables or databases (E) enables fine-grained control by separating sensitive information so users only access what they need. Together, these three approaches implement least privilege through access control, API restriction, and data isolation.

Why the other options are wrong

  • A. Disabling writes does not control read access or limit information exposure.
  • C. Encryption protects data in transit and at rest but does not restrict which users can access it.
  • F. Audit logging detects violations after they occur but does not prevent unauthorized access proactively.

Question 2

You are designing a basket abandonment system for an ecommerce company. The system will send a message to a user based on these rules: • No interaction by the user on the site for 1 hour Has added more than $30 worth of products to the basket • Has not completed a transaction You use Google Cloud Dataflow to process the data and decide if a message should be sent. How should you design the pipeline?

  1. Use a fixed-time window with a duration of 60 minutes.
  2. Use a sliding time window with a duration of 60 minutes.
  3. Use a session window with a gap time duration of 60 minutes.
  4. Use a global window with a time based trigger with a delay of 60 minutes.
Show answer and explanation

Correct answer: C. Use a session window with a gap time duration of 60 minutes.

A session window with a gap time duration of 60 minutes is the correct choice because it groups events based on user activity patterns. When a user has no interaction for 60 minutes (the gap duration), the session closes, triggering the abandonment logic. This naturally captures the requirement of no interaction for 1 hour. Session windows are designed for exactly this type of user behavior analysis where you need to detect inactivity periods and trigger actions when gaps occur.

Why the other options are wrong

  • A. Fixed windows emit results at fixed intervals regardless of user activity, missing the inactivity detection requirement.
  • B. Sliding windows would continuously overlap and re-evaluate, not specifically detecting a 60-minute gap in activity.
  • D. Global windows do not partition data by time and would not properly handle the 60-minute inactivity requirement.

Question 3

Your company handles data processing for a number of different clients. Each client prefers to use their own suite of analytics tools, with some allowing direct query access via Google BigQuery. You need to secure the data so that clients cannot see each other's data. You want to ensure appropriate access to the data. Which three steps should you take? (Choose three.)

  1. Load data into different partitions.
  2. Load data into a different dataset for each client.
  3. Put each client's BigQuery dataset into a different table.
  4. Restrict a client's dataset to approved users.
  5. Only allow a service account to access the datasets.
  6. Use the appropriate identity and access management (IAM) roles for each client's users.
Show answer and explanation

Correct answer: B, D, F. Load data into a different dataset for each client; Restrict a client's dataset to approved users; Use the appropriate identity and access management (IAM) roles for each client's users.

Load data into a different dataset for each client (B) provides logical separation, ensuring clients cannot see each other's data at the dataset level. Restrict a client's dataset to approved users (D) enforces access control so only authorized personnel from each client organization can access their data. Use appropriate IAM roles for each client's users (F) implements fine-grained permission management, assigning roles that match client responsibilities. Together, these three steps create a secure multi-tenant architecture with strong data isolation and access governance.

Why the other options are wrong

  • A. Partitions organize data within a single table and do not provide isolation between clients.
  • C. This mixes logical organization concepts; datasets already provide separation without needing multiple tables per client.
  • E. Service accounts are for applications, not users, and do not address client-specific access control needs.

Question 4

You want to process payment transactions in a point-of-sale application that will run on Google Cloud Platform. Your user base could grow exponentially, but you do not want to manage infrastructure scaling. Which Google database service should you use?

  1. Cloud SQL
  2. BigQuery
  3. Cloud Bigtable
  4. Cloud Datastore
Show answer and explanation

Correct answer: D. Cloud Datastore

Cloud Datastore is the correct choice for a point-of-sale application with potentially exponential user growth and no infrastructure management requirement. Datastore is a fully managed, serverless NoSQL database that automatically scales horizontally to handle increasing transaction volume without manual intervention. It provides ACID transactions suitable for payment processing and grows seamlessly as the user base expands.

Why the other options are wrong

  • A. Cloud SQL requires manual scaling decisions and infrastructure management as load increases.
  • B. BigQuery is designed for analytical queries, not transactional point-of-sale operations.
  • C. Cloud Bigtable requires manual cluster sizing and management, particularly for payment transaction consistency needs.

Question 5

You want to use a database of information about tissue samples to classify future tissue samples as either normal or mutated. You are evaluating an unsupervised anomaly detection method for classifying the tissue samples. Which two characteristic support this method? (Choose two.)

  1. There are very few occurrences of mutations relative to normal samples.
  2. There are roughly equal occurrences of both normal and mutated samples in the database.
  3. You expect future mutations to have different features from the mutated samples in the database.
  4. You expect future mutations to have similar features to the mutated samples in the database.
  5. You already have labels for which samples are mutated and which are normal in the database.
Show answer and explanation

Correct answer: A, C. There are very few occurrences of mutations relative to normal samples; You expect future mutations to have different features from the mutated samples in the database.

Unsupervised anomaly detection is most effective when mutations are rare relative to normal samples (A), as the algorithm learns normal patterns from the majority class and identifies deviations as anomalies. Additionally, if future mutations have different features from known mutations in the database (C), unsupervised methods work well because they detect novel patterns rather than requiring training on labeled examples of each mutation type. These characteristics align with the unsupervised approach's strength in identifying unexpected deviations.

Why the other options are wrong

  • B. Equal occurrence of both classes makes anomaly detection less effective; the algorithm needs a dominant normal class.
  • D. If future mutations resemble past mutations, supervised learning would be more appropriate and efficient.
  • E. Having labels indicates supervised learning is available, making it more suitable than unsupervised approaches.

Question 6

You need to store and analyze social media postings in Google BigQuery at a rate of 10,000 messages per minute in near real-time. Initially, design the application to use streaming inserts for individual postings. Your application also performs data aggregations right after the streaming inserts. You discover that the queries after streaming inserts do not exhibit strong consistency, and reports from the queries might miss in-flight data. How can you adjust your application design?

  1. Rewrite the application to load accumulated data every 2 minutes.
  2. Convert the streaming insert code to batch load for individual messages.
  3. Load the original message to Google Cloud SQL, and export the table every hour to BigQuery via streaming inserts.
  4. Estimate the average latency for data availability after streaming inserts, and always run queries after waiting twice as long.
Show answer and explanation

Correct answer: D. Estimate the average latency for data availability after streaming inserts, and always run queries after waiting twice as long.

Streaming inserts in BigQuery do not guarantee immediate strong consistency; data can take time to become available for queries. Estimating the average latency for data availability and always running queries after waiting twice as long (D) ensures that in-flight data has settled into the system before queries execute, improving result consistency and capturing all streamed records. This pragmatic approach accounts for BigQuery's eventual consistency model with streaming inserts without redesigning the pipeline.

Why the other options are wrong

  • A. Loading every 2 minutes introduces arbitrary delays and is less precise than accounting for actual latency.
  • B. Converting to batch loads would defeat the real-time streaming requirement.
  • C. Adding Cloud SQL as an intermediary introduces unnecessary complexity and does not solve the eventual consistency issue.

Question 7

Your startup has never implemented a formal security policy. Currently, everyone in the company has access to the datasets stored in Google BigQuery. Teams have freedom to use the service as they see fit, and they have not documented their use cases. You have been asked to secure the data warehouse. You need to discover what everyone is doing. What should you do first?

  1. Use Google Stackdriver Audit Logs to review data access.
  2. Get the identity and access management IIAM) policy of each table
  3. Use Stackdriver Monitoring to see the usage of BigQuery query slots.
  4. Use the Google Cloud Billing API to see what account the warehouse is being billed to.
Show answer and explanation

Correct answer: A. Use Google Stackdriver Audit Logs to review data access.

Google Stackdriver Audit Logs should be reviewed first to discover what data access patterns currently exist. These logs record who accessed which datasets, tables, and queries, providing visibility into actual usage patterns without modifying the system. This discovery phase is essential before implementing security policies, as it reveals how teams are using BigQuery and which data requires protection.

Why the other options are wrong

  • B. Checking IAM policies only shows current permissions, not actual usage patterns or who is accessing what.
  • C. Stackdriver Monitoring shows query slot usage but not which specific data is being accessed or by whom.
  • D. The billing API shows cost allocation but provides no insight into data access patterns or security vulnerabilities.

Question 8

Your company is migrating their 30-node Apache Hadoop cluster to the cloud. They want to re-use Hadoop jobs they have already created and minimize the management of the cluster as much as possible. They also want to be able to persist data beyond the life of the cluster. What should you do?

  1. Create a Google Cloud Dataflow job to process the data.
  2. Create a Google Cloud Dataproc cluster that uses persistent disks for HDFS.
  3. Create a Hadoop cluster on Google Compute Engine that uses persistent disks.
  4. Create a Cloud Dataproc cluster that uses the Google Cloud Storage connector.
  5. Create a Hadoop cluster on Google Compute Engine that uses Local SSD disks.
Show answer and explanation

Correct answer: D. Create a Cloud Dataproc cluster that uses the Google Cloud Storage connector.

Creating a Cloud Dataproc cluster that uses the Google Cloud Storage connector (D) addresses all requirements: Dataproc is a managed Hadoop service minimizing cluster management, the GCS connector allows reusing existing Hadoop jobs without modification, and Cloud Storage persists data beyond the cluster's lifetime. This approach provides the best balance of compatibility, minimal management overhead, and data durability.

Why the other options are wrong

  • A. Dataflow requires rewriting jobs in its pipeline model, not reusing existing Hadoop code.
  • B. Using persistent disks for HDFS ties data to the cluster, limiting portability and durability.
  • C. Building a custom cluster on Compute Engine requires manual infrastructure management.
  • E. Local SSD disks are ephemeral and do not persist data beyond cluster termination.

Question 9

Business owners at your company have given you a database of bank transactions. Each row contains the user ID, transaction type, transaction location, and transaction amount. They ask you to investigate what type of machine learning can be applied to the data. Which three machine learning applications can you use? (Choose three.)

  1. Supervised learning to determine which transactions are most likely to be fraudulent.
  2. Unsupervised learning to determine which transactions are most likely to be fraudulent.
  3. Clustering to divide the transactions into N categories based on feature similarity.
  4. Supervised learning to predict the location of a transaction.
  5. Reinforcement learning to predict the location of a transaction.
  6. Unsupervised learning to predict the location of a transaction.
Show answer and explanation

Correct answer: B, C, D. Unsupervised learning to determine which transactions are most likely to be fraudulent; Clustering to divide the transactions into N categories based on feature similarity; Supervised learning to predict the location of a transaction.

The data has no fraud label, so fraud detection must be unsupervised, for example flagging anomalous transactions. Clustering groups transactions into categories by similarity without labels. Location is a column in the data, so it can serve as the target for a supervised model that predicts it from the other features.

Why the other options are wrong

  • A. Supervised learning needs labelled examples of fraud, and the dataset has no fraud label.
  • E. Reinforcement learning is for agents learning actions from rewards, not for predicting a value in a static dataset.
  • F. Location is a known labelled column, so predicting it is a supervised problem, not an unsupervised one.

Question 10

Your company's on-premises Apache Hadoop servers are approaching end-of-life, and IT has decided to migrate the cluster to Google Cloud Dataproc. A like-for-like migration of the cluster would require 50 TB of Google Persistent Disk per node. The CIO is concerned about the cost of using that much block storage. You want to minimize the storage cost of the migration. What should you do?

  1. Put the data into Google Cloud Storage.
  2. Use preemptible virtual machines (VMs) for the Cloud Dataproc cluster.
  3. Tune the Cloud Dataproc cluster so that there is just enough disk for all data.
  4. Migrate some of the cold data into Google Cloud Storage, and keep only the hot data in Persistent Disk.
Show answer and explanation

Correct answer: A. Put the data into Google Cloud Storage.

Google Cloud Storage is significantly cheaper than Persistent Disk for large-scale data storage. Migrating the 50 TB per node to Cloud Storage reduces block storage costs while maintaining data accessibility for Dataproc processing. Dataproc can efficiently read from and write to Cloud Storage, making this the most cost-effective solution for the migration.

Why the other options are wrong

  • B. Preemptible VMs reduce compute costs but do not address the primary concern of expensive block storage.
  • C. Tuning disk allocation does not fundamentally solve the cost problem when 50 TB per node is required for operational data.
  • D. Splitting data between Persistent Disk and Cloud Storage adds operational complexity and still requires significant Persistent Disk investment for hot data.

What people say about the full Professional Data Engineer pack 4.6 from 7

Well, I still passed it. Amazing dump for Google
DennisProfessional Data Engineer
All those taking the Google Professional-Data-Engineer exam are advised to buy the exam questions from Cert Stash. Practising the similar exam first helps you score well in the real exam. I achieved 97% marks.
JeffreyProfessional Data Engineer
All Google questions are from your guide.
ElmaProfessional Data Engineer

Question 11

You need to set access to BigQuery for different departments within your company. Your solution should comply with the following requirements: • Each department should have access only to their data. • Each department will have one or more leads who need to be able to create and update tables and provide them to their team. • Each department has data analysts who need to be able to query but not modify data. How should you set access to the data in BigQuery?

  1. Create a dataset for each department. Assign the department leads the role of OWNER, and assign the data analysts the role of WRITER on their dataset.
  2. Create a dataset for each department. Assign the department leads the role of WRITER, and assign the data analysts the role of READER on their dataset.
  3. Create a table for each department. Assign the department leads the role of Owner, and assign the data analysts the role of Editor on the project the table is in.
  4. Create a table for each department. Assign the department leads the role of Editor, and assign the data analysts the role of Viewer on the project the table is in.
Show answer and explanation

Correct answer: B. Create a dataset for each department. Assign the department leads the role of WRITER, and assign the data analysts the role of READER on their dataset.

Dataset-level access control is the appropriate granularity for department-based data segregation in BigQuery. Department leads need the WRITER role to create, update, and manage tables within their dataset, while data analysts need the READER role to query data without modification capabilities. Using datasets (not tables) as the organizational unit allows proper isolation and permission management at the appropriate scope.

Why the other options are wrong

  • A. OWNER role is unnecessarily permissive for department leads and grants administrative privileges beyond what is required.
  • C. Table-level granularity is impractical for managing department access and does not provide proper isolation; project-level roles are too broad.
  • D. Assigning roles at the project level exposes data across departments and does not meet the isolation requirement.

Question 12

You are developing an application that uses a recommendation engine on Google Cloud. Your solution should display new videos to customers based on past views. Your solution needs to generate labels for the entities in videos that the customer has viewed. Your design must be able to provide very fast filtering suggestions based on data from other customer preferences on several TB of data. What should you do?

  1. Build and train a complex classification model with Spark MLlib to generate labels and filter the results. Deploy the models using Cloud Dataproc. Call the model from your application.
  2. Build and train a classification model with Spark MLlib to generate labels. Build and train a second classification model with Spark MLlib to filter results to match customer preferences. Deploy the models using Cloud Dataproc. Call the models from your application.
  3. Build an application that calls the Cloud Video Intelligence API to generate labels. Store data in Cloud Bigtable, and filter the predicted labels to match the user's viewing history to generate preferences.
  4. Build an application that calls the Cloud Video Intelligence API to generate labels. Store data in Cloud SQL, and join and filter the predicted labels to match the user's viewing history to generate preferences.
Show answer and explanation

Correct answer: C. Build an application that calls the Cloud Video Intelligence API to generate labels. Store data in Cloud Bigtable, and filter the predicted labels to match the user's viewing history to generate preferences.

Cloud Video Intelligence API efficiently generates labels for video content without building custom ML models. Cloud Bigtable provides fast filtering across several TB of data with very low latency, making it ideal for real-time recommendation queries. This combination of services directly addresses the need for fast label generation and rapid filtering of viewing history to provide recommendations.

Why the other options are wrong

  • A. Building and deploying complex Spark MLlib models via Dataproc adds unnecessary complexity when Video Intelligence API handles labeling more efficiently.
  • B. Training two separate Spark models is overly complex and doesn't leverage specialized GCP services designed for video analysis and fast filtering.
  • D. Cloud SQL cannot efficiently filter several TB of data with the speed required for real-time recommendations; Bigtable is designed for this use case.

Question 13

You are choosing a NoSQL database to handle telemetry data submitted from millions of Internet-of-Things (IoT) devices. The volume of data is growing at 100 TB per year, and each data entry has about 100 attributes. The data processing pipeline does not require atomicity, consistency, isolation, and durability (ACID). However, high availability and low latency are required. You need to analyze the data by querying against individual fields. Which three databases meet your requirements? (Choose three.)

  1. Redis
  2. HBase
  3. MySQL
  4. MongoDB
  5. Cassandra
  6. HDFS with Hive
Show answer and explanation

Correct answer: B, D, E. HBase; MongoDB; Cassandra

HBase, MongoDB, and Cassandra all meet the requirements. HBase is a distributed column-family store designed for massive scale, high availability, and low latency with no ACID requirements. MongoDB is a document database that supports querying individual fields, scales horizontally, and provides high availability. Cassandra is a distributed wide-column store optimized for high availability, massive scale, and low-latency writes with no ACID requirements. All three handle 100 TB+ scale and millions of concurrent devices well. Redis is an in-memory cache, not suitable for persistent 100 TB storage. MySQL is a relational database requiring ACID, which the requirements explicitly exclude. HDFS with Hive is designed for batch analytics, not low-latency queries on individual fields.

Why the other options are wrong

  • A. Redis is an in-memory cache unsuitable for persistent 100 TB IoT telemetry storage and high-latency batch processing.
  • C. MySQL is a relational database that requires ACID compliance, which is explicitly not needed; it also scales poorly for this distributed IoT scenario.
  • F. HDFS with Hive is a batch analytics platform, not optimized for low-latency queries or real-time data access on individual fields.

Question 14

You work for a manufacturing plant that batches application log files together into a single log file once a day at 2:00 AM. You have written a Google Cloud Dataflow job to process that log file. You need to make sure the log file in processed once per day as inexpensively as possible. What should you do?

  1. Change the processing job to use Google Cloud Dataproc instead.
  2. Manually start the Cloud Dataflow job each morning when you get into the office.
  3. Create a cron job with Google App Engine Cron Service to run the Cloud Dataflow job.
  4. Configure the Cloud Dataflow job as a streaming job so that it processes the log data immediately.
Show answer and explanation

Correct answer: C. Create a cron job with Google App Engine Cron Service to run the Cloud Dataflow job.

Creating a cron job with Google App Engine Cron Service is the most cost-effective and automated approach to trigger the Dataflow job once per day at a specific time (2:00 AM). App Engine Cron Service incurs minimal costs and automatically invokes the job without manual intervention. Option A would increase costs by switching to Dataproc. Option B requires manual intervention and is not scalable. Option D converts the job to streaming, which is more expensive and unnecessary for batch processing that occurs only once daily.

Why the other options are wrong

  • A. Dataproc is more expensive than Dataflow for this one-time daily batch operation.
  • B. Manual job triggering is not reliable, not scalable, and requires human intervention every day.
  • D. Streaming jobs are continuously running and more expensive than batch jobs; batch processing once daily does not require streaming architecture.

Question 15

You designed a database for patient records as a pilot project to cover a few hundred patients in three clinics. Your design used a single database table to represent all patients and their visits, and you used self-joins to generate reports. The server resource utilization was at 50%. Since then, the scope of the project has expanded. The database must now store 100 times more patient records. You can no longer run the reports, because they either take too long or they encounter errors with insufficient compute resources. How should you adjust the database design?

  1. Add capacity (memory and disk space) to the database server by the order of 200.
  2. Shard the tables into smaller ones based on date ranges, and only generate reports with prespecified date ranges.
  3. Normalize the master patient-record table into the patient table and the visits table, and create other necessary tables to avoid self-join.
  4. Partition the table into smaller tables, with one for each clinic. Run queries against the smaller table pairs, and use unions for consolidated reports.
Show answer and explanation

Correct answer: C. Normalize the master patient-record table into the patient table and the visits table, and create other necessary tables to avoid self-join.

The original denormalized design with self-joins creates severe performance problems at scale: redundant data storage, complex and inefficient self-join queries, and poor indexing opportunities. Normalizing the patient records into separate patient and visits tables eliminates data redundancy, enables efficient foreign key joins, supports better indexing, and dramatically improves query performance. This structural fix addresses root causes rather than symptoms. Simply adding server capacity (option A) is wasteful and insufficient for fundamentally inefficient queries. Sharding (option B) and partitioning (option D) address scalability but do not fix the underlying design flaw of self-joins and denormalization.

Why the other options are wrong

  • A. Adding 200x capacity is cost-prohibitive and does not solve inefficient self-join queries that will still be slow.
  • B. Sharding by date range helps with horizontal scaling but does not eliminate the inefficiency of self-joins on the same denormalized table.
  • D. Partitioning by clinic does not address the fundamental design problem of denormalization and self-joins, which remain inefficient within each partition.

Question 16

You have a data pipeline with a Dataflow job that aggregates and writes time series metrics to Bigtable. You notice that data is slow to update in Bigtable. This data feeds a dashboard used by thousands of users across the organization. You need to support additional concurrent users and reduce the amount of time required to write the data. Which two actions should you take? (Choose two.)

  1. Configure your Dataflow pipeline to use local execution
  2. Increase the maximum number of Dataflow workers by setting maxNumWorkers in PipelineOptions
  3. Increase the number of nodes in the Bigtable cluster
  4. Modify your Dataflow pipeline to use the Flatten transform before writing to Bigtable
  5. Modify your Dataflow pipeline to use the CoGroupByKey transform before writing to Bigtable
Show answer and explanation

Correct answer: B, C. Increase the maximum number of Dataflow workers by setting maxNumWorkers in PipelineOptions; Increase the number of nodes in the Bigtable cluster

Bigtable write throughput is the bottleneck, so increasing the number of Bigtable cluster nodes (option C) directly expands write capacity and reduces contention. Increasing Dataflow's maximum worker count (option B) enables the pipeline to process and aggregate metrics more quickly, reducing the time data spends in the pipeline before being written. This two-pronged approach improves both the processing speed and the write destination's capacity, directly addressing slow updates and supporting more concurrent dashboard users.

Why the other options are wrong

  • A. Local execution of Dataflow would eliminate distributed processing entirely, dramatically degrading performance and preventing scalability.
  • D. Flatten merely combines multiple PCollections and provides no benefit to write performance or data aggregation efficiency.
  • E. CoGroupByKey performs a grouping operation that doesn't address write latency or throughput to Bigtable; it may actually increase processing overhead.

Question 17

You have an Apache Kafka cluster on-prem with topics containing web application logs. You need to replicate the data to Google Cloud for analysis in BigQuery and Cloud Storage. The preferred replication method is mirroring to avoid deployment of Kafka Connect plugins. What should you do?

  1. Deploy a Kafka cluster on GCE VM Instances. Configure your on-prem cluster to mirror your topics to the cluster running in GCE. Use a Dataproc cluster or Dataflow job to read from Kafka and write to GCS.
  2. Deploy a Kafka cluster on GCE VM Instances with the Pub/Sub Kafka connector configured as a Sink connector. Use a Dataproc cluster or Dataflow job to read from Kafka and write to GCS.
  3. Deploy the Pub/Sub Kafka connector to your on-prem Kafka cluster and configure Pub/Sub as a Source connector. Use a Dataflow job to read from Pub/Sub and write to GCS.
  4. Deploy the Pub/Sub Kafka connector to your on-prem Kafka cluster and configure Pub/Sub as a Sink connector. Use a Dataflow job to read from Pub/Sub and write to GCS.
Show answer and explanation

Correct answer: A. Deploy a Kafka cluster on GCE VM Instances. Configure your on-prem cluster to mirror your topics to the cluster running in GCE. Use a Dataproc cluster or Dataflow job to read from Kafka and write to GCS.

The question explicitly states the preferred method is mirroring to avoid Kafka Connect plugins. Option A uses Kafka mirroring by deploying a Kafka cluster on GCE and configuring the on-prem cluster to mirror topics to it, then uses Dataproc or Dataflow to move data to GCS and BigQuery. This avoids plugin deployment while achieving the replication goal. The mirrored topics in the GCE Kafka cluster provide a clean source for downstream processing.

Why the other options are wrong

  • B. This option requires the Pub/Sub Kafka connector, which contradicts the requirement to avoid Kafka Connect plugins.
  • C. This option requires deploying the Pub/Sub Kafka connector to the on-prem cluster, violating the stated preference to avoid plugin deployment.
  • D. This option requires deploying the Pub/Sub Kafka connector to the on-prem cluster, violating the stated preference to avoid plugin deployment.

Question 18

You work for an economic consulting firm that helps companies identify economic trends as they happen. As part of your analysis, you use Google BigQuery to correlate customer data with the average prices of the 100 most common goods sold, including bread, gasoline, milk, and others. The average prices of these goods are updated every 30 minutes. You want to make sure this data stays up to date so you can combine it with other data in BigQuery as cheaply as possible. What should you do?

  1. Load the data every 30 minutes into a new partitioned table in BigQuery.
  2. Store and update the data in a regional Google Cloud Storage bucket and create a federated data source in BigQuery
  3. Store the data in Google Cloud Datastore. Use Google Cloud Dataflow to query BigQuery and combine the data programmatically with the data stored in Cloud Datastore
  4. Store the data in a file in a regional Google Cloud Storage bucket. Use Cloud Dataflow to query BigQuery and combine the data programmatically with the data stored in Google Cloud Storage.
Show answer and explanation

Correct answer: B. Store and update the data in a regional Google Cloud Storage bucket and create a federated data source in BigQuery

Storing the price data in a regional Cloud Storage bucket and creating a federated data source in BigQuery is the cheapest approach. Federated queries allow BigQuery to query data directly from Cloud Storage without loading it, avoiding storage costs in BigQuery itself while keeping data fresh. The prices are updated every 30 minutes in Cloud Storage, and BigQuery can access the latest version on each query. Option A creates unnecessary BigQuery storage overhead with partitioned tables. Option C adds Datastore and Dataflow costs unnecessarily. Option D uses Dataflow for data combination when BigQuery's native federated join capability is sufficient and cheaper.

Why the other options are wrong

  • A. Loading into a new partitioned table every 30 minutes creates excessive BigQuery storage costs and unnecessary data duplication.
  • C. Using Cloud Datastore and Dataflow adds significant cost and complexity when BigQuery federated queries provide a simpler, cheaper solution.
  • D. Using Dataflow to programmatically combine data incurs computation costs that federated queries in BigQuery avoid.

Question 19

Your company is currently setting up data pipelines for their campaign. For all the Google Cloud Pub/Sub streaming data, one of the important business requirements is to be able to periodically identify the inputs and their timings during their campaign. Engineers have decided to use windowing and transformation in Google Cloud Dataflow for this purpose. However, when testing this feature, they find that the Cloud Dataflow job fails for the all streaming insert. What is the most likely cause of this problem?

  1. They have not assigned the timestamp, which causes the job to fail
  2. They have not set the triggers to accommodate the data coming in late, which causes the job to fail
  3. They have not applied a global windowing function, which causes the job to fail when the pipeline is created
  4. They have not applied a non-global windowing function, which causes the job to fail when the pipeline is created
Show answer and explanation

Correct answer: D. They have not applied a non-global windowing function, which causes the job to fail when the pipeline is created

Streaming pipelines in Cloud Dataflow require a non-global windowing function to be applied. Without a specific windowing strategy (such as fixed, sliding, or session windows), the pipeline defaults to global windowing, which accumulates all data indefinitely and cannot emit results, causing the job to fail during pipeline creation. Global windowing is only suitable for bounded batch data, not unbounded streaming data.

Why the other options are wrong

  • A. While timestamps are important for windowing, they are not the root cause of job failure during pipeline creation; the issue is the missing windowing strategy itself.
  • B. Triggers handle late-arriving data but are not the primary cause of failure; the absence of a non-global windowing function is the fundamental issue.
  • C. Global windowing is the problem, not the solution; the error occurs because of the absence of a non-global windowing function.

Question 20

An online retailer has built their current application on Google App Engine. A new initiative at the company mandates that they extend their application to allow their customers to transact directly via the application. They need to manage their shopping transactions and analyze combined data from multiple datasets using a business intelligence (BI) tool. They want to use only a single database for this purpose. Which Google Cloud database should they choose?

  1. BigQuery
  2. Cloud SQL
  3. Cloud BigTable
  4. Cloud Datastore
Show answer and explanation

Correct answer: B. Cloud SQL

Cloud SQL is the appropriate choice for an online retailer needing to manage shopping transactions with ACID compliance and support for BI tool integration. It provides the relational database capabilities necessary for transaction management and can integrate with standard BI tools. A single Cloud SQL database can handle both operational transaction processing and analytical queries through its relational structure.

Why the other options are wrong

  • A. BigQuery is designed for analytical workloads, not transactional systems; it lacks the ACID guarantees and real-time transaction processing capabilities needed for shopping transactions.
  • C. Cloud Bigtable is a NoSQL wide-column store designed for time-series and analytical data, not for transactional integrity or BI tool integration.
  • D. Cloud Datastore is a NoSQL document database; while it can store transaction data, it is not the best choice for BI analysis and transactional processing.

Question 21

You create an important report for your large team in Google Data Studio 360. The report uses Google BigQuery as its data source. You notice that visualizations are not showing data that is less than 1 hour old. What should you do?

  1. Disable caching by editing the report settings.
  2. Disable caching in BigQuery by editing table details.
  3. Refresh your browser tab showing the visualizations.
  4. Clear your browser history for the past hour then reload the tab showing the virtualizations.
Show answer and explanation

Correct answer: A. Disable caching by editing the report settings.

Google Data Studio caches visualization data to improve performance and reduce query costs. When data is less than 1 hour old, the cached results prevent real-time data from appearing in reports. Disabling caching in Data Studio's report settings ensures that queries run fresh against BigQuery without stale cached results. This is the proper configuration point for controlling report freshness. Browser cache clearing (option D) does not affect BigQuery caching. BigQuery table-level caching (option B) is not the appropriate control for this scenario. Refreshing the browser (option C) will still retrieve cached Data Studio results.

Why the other options are wrong

  • B. BigQuery table caching is not the control point for Data Studio report freshness.
  • C. Refreshing the browser will still use Data Studio's cache, not bypass it.
  • D. Clearing browser history does not affect Data Studio's server-side caching mechanism.

Question 22

You're training a model to predict housing prices based on an available dataset with real estate properties. Your plan is to train a fully connected neural net, and you've discovered that the dataset contains latitude and longitude of the property. Real estate professionals have told you that the location of the property is highly influential on price, so you'd like to engineer a feature that incorporates this physical dependency. What should you do?

  1. Provide latitude and longitude as input vectors to your neural net.
  2. Create a numeric column from a feature cross of latitude and longitude.
  3. Create a feature cross of latitude and longitude, bucketize it at the minute level and use L1 regularization during optimization.
  4. Create a feature cross of latitude and longitude, bucketize it at the minute level and use L2 regularization during optimization.
Show answer and explanation

Correct answer: C. Create a feature cross of latitude and longitude, bucketize it at the minute level and use L1 regularization during optimization.

Creating a feature cross of latitude and longitude captures spatial relationships, and bucketizing at the minute level (approximately 1-2 km resolution) groups nearby locations into discrete geographic regions relevant to real estate pricing. L1 regularization encourages sparsity, which is appropriate for high-dimensional crossed features by automatically zeroing out less important location buckets, making the model more interpretable and preventing overfitting on the geographic feature.

Why the other options are wrong

  • A. Providing raw latitude and longitude coordinates doesn't capture the non-linear geographic relationships that influence price; a neural net can learn some patterns but won't effectively model local geographic clustering.
  • B. Creating a numeric column from a feature cross without bucketization produces a high-cardinality continuous feature that doesn't effectively group locations and loses the geographic clustering benefits.
  • D. L2 regularization shrinks weights rather than encouraging sparsity, making it less effective than L1 for high-dimensional crossed features where many buckets may be irrelevant.

Question 23

You are designing storage for very large text files for a data pipeline on Google Cloud. You want to support ANSI SQL queries. You also want to support compression and parallel load from the input locations using Google recommended practices. What should you do?

  1. Transform text files to compressed Avro using Cloud Dataflow. Use BigQuery for storage and query.
  2. Transform text files to compressed Avro using Cloud Dataflow. Use Cloud Storage and BigQuery permanent linked tables for query.
  3. Compress text files to gzip using the Grid Computing Tools. Use BigQuery for storage and query.
  4. Compress text files to gzip using the Grid Computing Tools. Use Cloud Storage, and then import into Cloud Bigtable for query.
Show answer and explanation

Correct answer: A. Transform text files to compressed Avro using Cloud Dataflow. Use BigQuery for storage and query.

For very large text files in a data pipeline requiring ANSI SQL queries with compression and parallel loading, the optimal approach is to transform text files to compressed Avro format using Cloud Dataflow and store directly in BigQuery. BigQuery natively supports Avro format, provides excellent compression, enables parallel loading, and is Google's recommended practice for analytical SQL workloads. BigQuery itself handles the storage and provides the query engine without requiring additional linked table configurations.

Why the other options are wrong

  • B. Linked tables add unnecessary complexity; BigQuery can store and query Avro data directly without requiring permanent linked table setup.
  • C. Gzip compression is less efficient for structured data than Avro, and Grid Computing Tools is not a Google Cloud recommended service for this task.
  • D. Cloud Bigtable is designed for key-value workloads and real-time operations, not for analytical SQL queries on large text files, making it unsuitable for this use case.

Question 24

Your globally distributed auction application allows users to bid on items. Occasionally, users place identical bids at nearly identical times, and different application servers process those bids. Each bid event contains the item, amount, user, and timestamp. You want to collate those bid events into a single location in real time to determine which user bid first. What should you do?

  1. Create a file on a shared file and have the application servers write all bid events to that file. Process the file with Apache Hadoop to identify which user bid first.
  2. Have each application server write the bid events to Cloud Pub/Sub as they occur. Push the events from Cloud Pub/Sub to a custom endpoint that writes the bid event information into Cloud SQL.
  3. Set up a MySQL database for each application server to write bid events into. Periodically query each of those distributed MySQL databases and update a master MySQL database with bid event information.
  4. Have each application server write the bid events to Google Cloud Pub/Sub as they occur. Use a pull subscription to pull the bid events using Google Cloud Dataflow. Give the bid for each item to the user in the bid event that is processed first.
Show answer and explanation

Correct answer: D. Have each application server write the bid events to Google Cloud Pub/Sub as they occur. Use a pull subscription to pull the bid events using Google Cloud Dataflow. Give the bid for each item to the user in the bid event that is processed first.

This solution uses Cloud Pub/Sub to collect bid events from distributed application servers in real time and Cloud Dataflow with a pull subscription to process them sequentially. Dataflow maintains event ordering and processes events in the order they arrive, allowing the system to correctly identify which user bid first based on processing order. This architecture handles global distribution, real-time requirements, and maintains event ordering.

Why the other options are wrong

  • A. Using a shared file and Hadoop is not suitable for real-time processing and does not handle the ordering requirements for nearly simultaneous bids across distributed servers.
  • B. While Pub/Sub is correct, pushing to a custom endpoint and Cloud SQL does not guarantee proper ordering or handle the requirement to determine which bid arrived first.
  • C. Distributed MySQL databases with periodic synchronization do not provide real-time processing and cannot reliably determine which bid occurred first across geographically distributed systems.

Question 25

You are designing the database schema for a machine learning-based food ordering service that will predict what users want to eat. Here is some of the information you need to store: • The user profile: What the user likes and doesn't like to eat • The user account information: Name, address, preferred meal times • The order information: When orders are made, from where, to whom The database will be used to store all the transactional data of the product. You want to optimize the data schema. Which Google Cloud Platform product should you use?

  1. BigQuery
  2. Cloud SQL
  3. Cloud Bigtable
  4. Cloud Datastore
Show answer and explanation

Correct answer: B. Cloud SQL

Cloud SQL is the appropriate choice for this transactional food ordering database. The requirements include structured user profiles, account information, and order details that benefit from relational schema design with ACID guarantees for transaction integrity. Cloud SQL supports complex queries across related tables and maintains data consistency, which is essential for an ordering system. BigQuery is optimized for analytical queries, not transactional workloads. Cloud Bigtable is designed for time-series and wide-column data at massive scale, not structured relational data. Cloud Datastore is suitable for unstructured document data but lacks the relational capabilities needed here.

Why the other options are wrong

  • A. BigQuery is an analytical data warehouse, not a transactional database; it is inefficient for frequent inserts and updates in an ordering system.
  • C. Cloud Bigtable is designed for time-series analytics and sparse data at massive scale, not structured relational transactional data.
  • D. Cloud Datastore is optimized for unstructured documents and lacks the relational schema and ACID transaction support needed for an ordering system.

25 questions in. Want the rest? The first half of the pack, free by email · The full pack

Question 26

You architect a system to analyze seismic data. Your extract, transform, and load (ETL) process runs as a series of MapReduce jobs on an Apache Hadoop cluster. The ETL process takes days to process a data set because some steps are computationally expensive. Then you discover that a sensor calibration step has been omitted. How should you change your ETL process to carry out sensor calibration systematically in the future?

  1. Modify the transformMapReduce jobs to apply sensor calibration before they do anything else.
  2. Introduce a new MapReduce job to apply sensor calibration to raw data, and ensure all other MapReduce jobs are chained after this.
  3. Add sensor calibration data to the output of the ETL process, and document that all users need to apply sensor calibration themselves.
  4. Develop an algorithm through simulation to predict variance of data output from the last MapReduce job based on calibration factors, and apply the correction to all data.
Show answer and explanation

Correct answer: B. Introduce a new MapReduce job to apply sensor calibration to raw data, and ensure all other MapReduce jobs are chained after this.

Introducing a dedicated MapReduce job for sensor calibration as the first step in the pipeline ensures that all downstream jobs receive calibrated data. This approach systematically applies calibration to raw data before any transformations occur, making the calibration step reusable and maintainable. Chaining this calibration job before all other MapReduce jobs ensures consistency and correctness throughout the ETL process.

Why the other options are wrong

  • A. Modifying existing transformation jobs to include calibration duplicates logic and is harder to maintain; a separate dedicated job is cleaner and more modular.
  • C. Adding calibration data to output and requiring users to apply it themselves is unreliable and shifts responsibility away from the ETL process.
  • D. Predicting variance through simulation and applying corrections to output is less accurate than performing actual sensor calibration on raw input data.

Question 27

Your company is streaming real-time sensor data from their factory floor into Bigtable and they have noticed extremely poor performance. How should the row key be redesigned to improve Bigtable performance on queries that populate real-time dashboards?

  1. Use a row key of the form <timestamp>.
  2. Use a row key of the form <sensorid>.
  3. Use a row key of the form <timestamp>#<sensorid>.
  4. Use a row key of the form >#<sensorid>#<timestamp>.
Show answer and explanation

Correct answer: D. Use a row key of the form >#<sensorid>#<timestamp>.

The row key format of >#<sensorid>#<timestamp> uses a reverse timestamp prefix (indicated by > to show reverse lexicographic ordering) combined with sensor ID. This design distributes writes evenly across the row key space because data from different sensors at different times spreads across multiple tablets, preventing hotspotting. The reverse timestamp ensures that recent data (which is queried frequently for dashboards) is stored together while avoiding the sequential timestamp pattern that would concentrate all writes to a single tablet. The sensor ID component further distributes the data.

Why the other options are wrong

  • A. Using only timestamp creates a hotspot problem because all new writes go to the end of the key range, concentrating load on a single tablet server.
  • B. Using only sensor ID means all data from one sensor is in the same tablet, and as multiple sensors write simultaneously, writes are not distributed evenly.
  • C. The format <timestamp>#<sensorid> still has the hotspot issue of sequential timestamps, causing poor write performance even though sensor ID is included.

Question 28

You are working on a niche product in the image recognition domain. Your team has developed a model that is dominated by custom C++ TensorFlow ops your team has implemented. These ops are used inside your main training loop and are performing bulky matrix multiplications. It currently takes up to several days to train a model. You want to decrease this time significantly and keep the cost low by using an accelerator on Google Cloud. What should you do?

  1. Use Cloud TPUs without any additional adjustment to your code.
  2. Use Cloud TPUs after implementing GPU kernel support for your customs ops.
  3. Use Cloud GPUs after implementing GPU kernel support for your customs ops.
  4. Stay on CPUs, and increase the size of the cluster you're training your model on.
Show answer and explanation

Correct answer: C. Use Cloud GPUs after implementing GPU kernel support for your customs ops.

Your custom C++ TensorFlow ops require GPU kernel implementation to run on accelerators. Cloud TPUs cannot execute custom CUDA kernels and would not provide speedup without rewriting the custom ops. Cloud GPUs support CUDA kernels, allowing your team to implement GPU-optimized versions of the custom ops. This approach provides significant training speedup while maintaining cost efficiency compared to scaling up CPU clusters.

Why the other options are wrong

  • A. TPUs cannot execute custom C++ TensorFlow ops without GPU kernel support, making this ineffective.
  • B. While TPUs are powerful, they require rewriting custom ops as TPU kernels, which is more complex than GPU kernels for custom code.
  • D. CPU cluster scaling would be extremely expensive and slow compared to GPU acceleration for matrix multiplication workloads.

Question 29

An external customer provides you with a daily dump of data from their database. The data flows into Google Cloud Storage GCS as comma-separated values (CSV) files. You want to analyze this data in Google BigQuery, but the data could have rows that are formatted incorrectly or corrupted. How should you build this pipeline?

  1. Use federated data sources, and check data in the SQL query.
  2. Enable BigQuery monitoring in Google Stackdriver and create an alert.
  3. Import the data into BigQuery using the gcloud CLI and set max_bad_records to 0.
  4. Run a Google Cloud Dataflow batch pipeline to import the data into BigQuery, and push errors to another dead-letter table for analysis.
Show answer and explanation

Correct answer: D. Run a Google Cloud Dataflow batch pipeline to import the data into BigQuery, and push errors to another dead-letter table for analysis.

A production data pipeline handling external CSV files with potential corruption requires robust error handling and visibility. Google Cloud Dataflow provides a managed batch processing service that can parse CSV files, validate records, and gracefully handle errors. Using a dead-letter table (a separate table for rejected records) enables data quality analysis without blocking the entire pipeline. This approach separates bad data from good data for investigation. Federated queries (option A) don't validate data and would fail on corrupted rows. Stackdriver monitoring (option B) only alerts after problems occur. Setting max_bad_records to 0 (option C) would reject all bad data silently without capturing it for analysis.

Why the other options are wrong

  • A. Federated queries on raw CSV files do not validate or handle corrupted records; they will fail on bad data.
  • B. Monitoring and alerts only notify of problems but do not handle or recover from corrupted data.
  • C. Setting max_bad_records to 0 rejects all bad records silently without capturing them for analysis or quality review.

Question 30

You want to automate execution of a multi-step data pipeline running on Google Cloud. The pipeline includes Dataproc and Dataflow jobs that have multiple dependencies on each other. You want to use managed services where possible, and the pipeline will run every day. Which tool should you use?

  1. cron
  2. Cloud Composer
  3. Cloud Scheduler
  4. Workflow Templates on Dataproc
Show answer and explanation

Correct answer: B. Cloud Composer

Cloud Composer is the managed orchestration service purpose-built for scheduling and managing multi-step data pipelines with complex dependencies on Google Cloud. It uses Apache Airflow to define workflows as Directed Acyclic Graphs (DAGs), allowing declarative specification of dependencies between Dataproc and Dataflow jobs. Composer is a fully managed service that runs daily and provides native support for monitoring, retries, and error handling across heterogeneous job types, making it the ideal choice for this use case.

Why the other options are wrong

  • A. cron is a basic scheduling tool that cannot model complex job dependencies or provide the orchestration capabilities needed for multi-step pipelines.
  • C. Cloud Scheduler is for simple time-based scheduling of individual tasks, not for managing complex multi-job workflows with dependencies.
  • D. Workflow Templates on Dataproc are specific to submitting jobs to Dataproc clusters and cannot orchestrate dependencies with external services like Dataflow.

Question 31

Your company is loading comma-separated values (CSV) files into Google BigQuery. The data is fully imported successfully; however, the imported data is not matching byte-to-byte to the source file. What is the most likely cause of this problem?

  1. The CSV data loaded in BigQuery is not flagged as CSV.
  2. The CSV data has invalid rows that were skipped on import.
  3. The CSV data loaded in BigQuery is not using BigQuery's default encoding.
  4. The CSV data has not gone through an ETL phase before loading into BigQuery.
Show answer and explanation

Correct answer: C. The CSV data loaded in BigQuery is not using BigQuery's default encoding.

The most likely cause of byte-to-byte mismatch between source CSV and imported data is encoding differences. If the CSV source file uses a different encoding (such as UTF-8 with BOM, Latin-1, or other encodings) and BigQuery defaults to UTF-8 without BOM or uses a different default encoding, the byte representation will differ even though the data appears correct. BigQuery's default encoding assumption may not match the source file's actual encoding, causing byte-level differences. Option A is incorrect because the data format flag doesn't affect bytes. Option B would typically cause row-level errors or rejections. Option D is unnecessary for a successful import.

Why the other options are wrong

  • A. The CSV flag designation doesn't affect byte-level representation of successfully imported data.
  • B. Invalid rows that are skipped would result in fewer rows, not byte-to-byte mismatch in imported rows.
  • D. An ETL phase is not necessary to explain byte-level encoding differences in a successful import.

Question 32

You are deploying 10,000 new Internet of Things devices to collect temperature data in your warehouses globally. You need to process, store and analyze these very large datasets in real time. What should you do?

  1. Send the data to Google Cloud Datastore and then export to BigQuery.
  2. Send the data to Google Cloud Pub/Sub, stream Cloud Pub/Sub to Google Cloud Dataflow, and store the data in Google BigQuery.
  3. Send the data to Cloud Storage and then spin up an Apache Hadoop cluster as needed in Google Cloud Dataproc whenever analysis is required.
  4. Export logs in batch to Google Cloud Storage and then spin up a Google Cloud SQL instance, import the data from Cloud Storage, and run an analysis as needed.
Show answer and explanation

Correct answer: B. Send the data to Google Cloud Pub/Sub, stream Cloud Pub/Sub to Google Cloud Dataflow, and store the data in Google BigQuery.

This architecture is purpose-built for real-time streaming IoT scenarios: Pub/Sub ingests data from 10,000 devices at scale, Dataflow provides serverless stream processing for transformations and aggregations, and BigQuery provides real-time storage and analysis of the large datasets. This combination ensures low-latency processing, automatic scaling, and immediate queryability of results.

Why the other options are wrong

  • A. Datastore is not optimized for this scale of real-time ingestion, and batch export to BigQuery introduces latency incompatible with real-time requirements.
  • C. Batch processing with Hadoop clusters cannot meet real-time processing needs and requires manual cluster provisioning.
  • D. Batch log export and on-demand SQL instance provisioning introduces unacceptable latency and manual overhead for real-time analytics.

Question 33

You are building a model to predict whether or not it will rain on a given day. You have thousands of input features and want to see if you can improve training speed by removing some features while having a minimum effect on model accuracy. What can you do?

  1. Eliminate features that are highly correlated to the output labels.
  2. Combine highly co-dependent features into one representative feature.
  3. Instead of feeding in each feature individually, average their values in batches of 3.
  4. Remove the features that have null values for more than 50% of the training records.
Show answer and explanation

Correct answer: B. Combine highly co-dependent features into one representative feature.

Combining highly co-dependent (multicollinear) features into a single representative feature reduces dimensionality while preserving the underlying information. This technique improves training speed by reducing the number of features the model must process, and it minimizes accuracy loss because multicollinear features are redundant, they provide overlapping information. This is a standard feature engineering approach in machine learning, distinct from feature selection which removes features entirely.

Why the other options are wrong

  • A. Eliminating features highly correlated with output labels would remove the most predictive features, severely harming model accuracy rather than maintaining it.
  • D. Removing features with more than 50% null values may discard valuable information and doesn't specifically address the goal of reducing training time while maintaining accuracy.
  • C. Arbitrarily averaging feature values in batches of 3 destroys feature information and relationships without principled justification, likely degrading model performance.

Question 34

You have a petabyte of analytics data and need to design a storage and processing platform for it. You must be able to perform data warehousestyle analytics on the data in Google Cloud and expose the dataset as files for batch analysis tools in other cloud providers. What should you do?

  1. Store and process the entire dataset in BigQuery.
  2. Store and process the entire dataset in Bigtable.
  3. Store the full dataset in BigQuery, and store a compressed copy of the data in a Cloud Storage bucket.
  4. Store the warm data as files in Cloud Storage, and store the active data in BigQuery. Keep this ratio as 80% warm and 20% active.
Show answer and explanation

Correct answer: C. Store the full dataset in BigQuery, and store a compressed copy of the data in a Cloud Storage bucket.

Storing the full dataset in BigQuery enables high-performance warehouse-style analytics within Google Cloud, while simultaneously storing a compressed copy in Cloud Storage provides the files needed for batch analysis tools in other cloud providers. This dual-storage approach satisfies both requirements: BigQuery optimizes analytics performance for Google Cloud workloads, and Cloud Storage provides portable, accessible files that can be retrieved and processed by external tools without vendor lock-in.

Why the other options are wrong

  • A. BigQuery alone does not expose data as files accessible to tools in other cloud providers without additional export steps.
  • B. Bigtable is designed for real-time operational workloads, not warehouse-style analytics, and does not support the analytical query patterns needed.
  • D. An 80/20 split is arbitrary and unnecessarily complex; the 20% active data in BigQuery is insufficient for comprehensive analytics on a petabyte dataset.

Question 35

Your company needs to upload their historic data to Cloud Storage. The security rules don't allow access from external IPs to their on-premises resources. After an initial upload, they will add new data from existing on-premises applications every day. What should they do?

  1. Execute gsutil rsync from the on-premises servers.
  2. Use Dataflow and write the data to Cloud Storage.
  3. Write a job template in Dataproc to perform the data transfer.
  4. Install an FTP server on a Compute Engine VM to receive the files and move them to Cloud Storage.
Show answer and explanation

Correct answer: A. Execute gsutil rsync from the on-premises servers.

gsutil rsync runs on the on-premises servers themselves, eliminating the need for external IP access to on-premises resources. The security rules only restrict external access to on-premises systems, not outbound connections from on-premises to Google Cloud. This approach is direct, efficient, and repeatable for daily incremental uploads.

Why the other options are wrong

  • B. Dataflow requires either on-premises Dataflow runner setup or external network access to on-premises resources, both violating security constraints.
  • C. Dataproc job templates still require network connectivity from Google Cloud to on-premises applications, which security rules prohibit.
  • D. Installing an FTP server on a Compute Engine VM requires data to be transferred externally from on-premises to the VM, violating the security rules against external IP access.

Question 36

You're using Bigtable for a real-time application, and you have a heavy load that is a mix of read and writes. You've recently identified an additional use case and need to perform hourly an analytical job to calculate certain statistics across the whole database. You need to ensure both the reliability of your production application as well as the analytical workload. What should you do?

  1. Export Bigtable dump to GCS and run your analytical job on top of the exported files.
  2. Add a second cluster to an existing instance with a multi-cluster routing, use live-traffic app profile for your regular workload and batchanalytics profile for the analytics workload.
  3. Add a second cluster to an existing instance with a single-cluster routing, use live-traffic app profile for your regular workload and batchanalytics profile for the analytics workload.
  4. Increase the size of your existing cluster twice and execute your analytics workload on your new resized cluster.
Show answer and explanation

Correct answer: C. Add a second cluster to an existing instance with a single-cluster routing, use live-traffic app profile for your regular workload and batchanalytics profile for the analytics workload.

Adding a second cluster and using single-cluster routing app profiles pins each workload to its own cluster: live traffic to one, batch analytics to the other. Replication keeps the data in both, so the hourly analytics job never competes for resources with the production application.

Why the other options are wrong

  • A. Exporting the entire database every hour is slow and the export itself puts load on the production cluster.
  • B. Multi-cluster routing can send requests to any cluster, so analytics and live traffic could land on the same cluster and interfere.
  • D. Resizing a single cluster still has both workloads sharing the same nodes, so the analytics job can still affect production.

Question 37

Your team is responsible for developing and maintaining ETLs in your company. One of your Dataflow jobs is failing because of some errors in the input data, and you need to improve reliability of the pipeline (incl. being able to reprocess all failing data). What should you do?

  1. Add a filtering step to skip these types of errors in the future, extract erroneous rows from logs.
  2. Add a try"¦ catch block to your DoFn that transforms the data, extract erroneous rows from logs.
  3. Add a try"¦ catch block to your DoFn that transforms the data, write erroneous rows to Pub/Sub PubSub directly from the DoFn.
  4. Add a try"¦ catch block to your DoFn that transforms the data, use a sideOutput to create a PCollection that can be stored to Pub/Sub later.
Show answer and explanation

Correct answer: D. Add a try"¦ catch block to your DoFn that transforms the data, use a sideOutput to create a PCollection that can be stored to Pub/Sub later.

Option D uses a try-catch block to handle errors and a sideOutput to create a separate PCollection containing erroneous rows. This approach allows errors to be captured, stored persistently to Pub/Sub, and reprocessed later as needed. Side outputs in Dataflow enable splitting data streams based on conditions while maintaining proper error handling semantics and enabling reliable reprocessing of failed records.

Why the other options are wrong

  • A. Filtering errors only skips problematic data without capturing it for reprocessing, losing the ability to handle failures reliably.
  • B. Writing directly to logs loses the data in an unstructured format that cannot be reliably reprocessed as a pipeline input.
  • C. Writing directly to Pub/Sub from within a DoFn violates Dataflow's model and can cause issues with exactly-once processing guarantees and state management.

Question 38

You are designing an Apache Beam pipeline to enrich data from Cloud Pub/Sub with static reference data from BigQuery. The reference data is small enough to fit in memory on a single worker. The pipeline should write enriched results to BigQuery for analysis. Which job type and transforms should this pipeline use?

  1. Batch job, PubSubIO, side-inputs
  2. Streaming job, PubSubIO, JdbcIO, side-outputs
  3. Streaming job, PubSubIO, BigQueryIO, side-inputs
  4. Streaming job, PubSubIO, BigQueryIO, side-outputs
Show answer and explanation

Correct answer: C. Streaming job, PubSubIO, BigQueryIO, side-inputs

This pipeline requires a streaming job because data continuously arrives from Cloud Pub/Sub. PubSubIO reads from Pub/Sub, and BigQueryIO writes enriched results to BigQuery. Side-inputs are the correct transform for joining small, static reference data (the BigQuery table) with streaming data on each worker, avoiding the need to reload reference data repeatedly.

Why the other options are wrong

  • A. Batch jobs don't process continuous Pub/Sub streams; they work with bounded data sources.
  • B. JdbcIO is for relational databases and unnecessary here; side-outputs are for splitting results, not enriching with reference data.
  • D. Side-outputs split a single input into multiple outputs, not enrich data with reference information.

Question 39

You have a data pipeline that writes data to Cloud Bigtable using well-designed row keys. You want to monitor your pipeline to determine when to increase the size of your Cloud Bigtable cluster. Which two actions can you take to accomplish this? (Choose two.)

  1. Review Key Visualizer metrics. Increase the size of the Cloud Bigtable cluster when the Read pressure index is above 100.
  2. Review Key Visualizer metrics. Increase the size of the Cloud Bigtable cluster when the Write pressure index is above 100.
  3. Monitor the latency of write operations. Increase the size of the Cloud Bigtable cluster when there is a sustained increase in write latency.
  4. Monitor storage utilization. Increase the size of the Cloud Bigtable cluster when utilization increases above 70% of max capacity.
  5. Monitor latency of read operations. Increase the size of the Cloud Bigtable cluster of read operations take longer than 100 ms.
Show answer and explanation

Correct answer: A, C. Review Key Visualizer metrics. Increase the size of the Cloud Bigtable cluster when the Read pressure index is above 100; Monitor the latency of write operations. Increase the size of the Cloud Bigtable cluster when there is a sustained increase in write latency.

Key Visualizer metrics provide pressure indices (Read and Write) that indicate when a cluster needs scaling; indices above 100 signal insufficient capacity. Additionally, monitoring write latency for sustained increases helps identify when the cluster is saturated and needs expansion. These two approaches directly measure cluster stress and performance degradation.

Why the other options are wrong

  • B. Write pressure index is less reliable than read pressure for scaling decisions; Key Visualizer's pressure indices emphasize read operations.
  • D. Storage utilization alone doesn't indicate performance bottlenecks; a cluster can have high throughput issues before hitting storage limits.
  • E. A 100 ms threshold for read latency is arbitrary and doesn't reflect actual Bigtable performance requirements; latency patterns vary by use case.

Question 40

You are building a new application that you need to collect data from in a scalable way. Data arrives continuously from the application throughout the day, and you expect to generate approximately 150 GB of JSON data per day by the end of the year. Your requirements are: • Decoupling producer from consumer • Space and cost-efficient storage of the raw ingested data, which is to be stored indefinitely • Near real-time SQL query • Maintain at least 2 years of historical data, which will be queried with SQL Which pipeline should you use to meet these requirements?

  1. Create an application that provides an API. Write a tool to poll the API and write data to Cloud Storage as gzipped JSON files.
  2. Create an application that writes to a Cloud SQL database to store the data. Set up periodic exports of the database to write to Cloud Storage and load into BigQuery.
  3. Create an application that publishes events to Cloud Pub/Sub, and create Spark jobs on Cloud Dataproc to convert the JSON data to Avro format, stored on HDFS on Persistent Disk.
  4. Create an application that publishes events to Cloud Pub/Sub, and create a Cloud Dataflow pipeline that transforms the JSON event payloads to Avro, writing the data to Cloud Storage and BigQuery.
Show answer and explanation

Correct answer: D. Create an application that publishes events to Cloud Pub/Sub, and create a Cloud Dataflow pipeline that transforms the JSON event payloads to Avro, writing the data to Cloud Storage and BigQuery.

Cloud Pub/Sub decouples the application from consumers and handles continuous data ingestion at scale. Cloud Dataflow provides a managed, scalable processing layer that transforms JSON to Avro format, enabling cost-efficient storage. Writing to Cloud Storage provides indefinite, compressed storage for raw data, while simultaneously writing to BigQuery enables near real-time SQL queries. BigQuery natively supports 2+ years of historical data retention with SQL access, meeting all requirements including cost efficiency, decoupling, real-time querying, and historical retention.

Why the other options are wrong

  • A. Polling an API is inefficient for continuous data and doesn't decouple producer from consumer; this approach lacks real-time processing capabilities.
  • B. Cloud SQL is not designed for 150 GB daily volumes and doesn't scale well for continuous ingestion; periodic exports introduce latency and don't provide near real-time SQL access.
  • C. Dataproc with HDFS on Persistent Disk is expensive for raw data storage and doesn't provide native SQL query capabilities; this approach lacks the scalability and query performance of BigQuery.

Question 41

You work on a regression problem in a natural language processing domain, and you have 100M labeled examples in your dataset. You have randomly shuffled your data and split your dataset into train and test samples (in a 90/10 ratio). After you trained the neural network and evaluated your model on a test set, you discover that the root-mean-squared error (RMSE) of your model is twice as high on the train set as on the test set. How should you improve the performance of your model?

  1. Increase the share of the test sample in the train-test split.
  2. Try to collect more data and increase the size of your dataset.
  3. Try out regularization techniques (e.g., dropout of batch normalization) to avoid overfitting.
  4. Increase the complexity of your model by, e.g., introducing an additional layer or increase sizing the size of vocabularies or n-grams used.
Show answer and explanation

Correct answer: D. Increase the complexity of your model by, e.g., introducing an additional layer or increase sizing the size of vocabularies or n-grams used.

Training error that is higher than test error means the model is not even fitting the training data well, which signals underfitting rather than overfitting. The fix is more model capacity, such as extra layers or larger vocabularies and n-grams, so it can capture more of the signal in the 100M examples.

Why the other options are wrong

  • A. Changing the split ratio does not improve how well the model learns and only reduces training data.
  • B. With 100M examples, more data is not the bottleneck for an underfitting model.
  • C. Regularisation fights overfitting and would make an underfitting model worse.

Question 42

You are deploying MariaDB SQL databases on GCE VM Instances and need to configure monitoring and alerting. You want to collect metrics including network connections, disk IO and replication status from MariaDB with minimal development effort and use StackDriver for dashboards and alerts. What should you do?

  1. Install the OpenCensus Agent and create a custom metric collection application with a StackDriver exporter.
  2. Place the MariaDB instances in an Instance Group with a Health Check.
  3. Install the StackDriver Logging Agent and configure fluentd in_tail plugin to read MariaDB logs.
  4. Install the StackDriver Agent and configure the MySQL plugin.
Show answer and explanation

Correct answer: D. Install the StackDriver Agent and configure the MySQL plugin.

The StackDriver Agent with the MySQL plugin provides minimal development effort for collecting MariaDB metrics including network connections, disk IO, and replication status. Since MariaDB is MySQL-compatible, the MySQL plugin works directly. The agent automatically exposes these metrics to StackDriver for dashboards and alerting without requiring custom metric collection code or manual log parsing.

Why the other options are wrong

  • A. OpenCensus Agent requires developing a custom metric collection application, which violates the minimal development effort requirement.
  • B. Instance Groups and Health Checks provide basic instance-level monitoring but cannot collect MariaDB-specific metrics like replication status or disk IO.
  • C. StackDriver Logging Agent with fluentd reads logs but does not directly collect the required database metrics like network connections and replication status in a structured way for alerting.

Question 43

Your company is performing data preprocessing for a learning algorithm in Google Cloud Dataflow. Numerous data logs are being are being generated during this step, and the team wants to analyze them. Due to the dynamic nature of the campaign, the data is growing exponentially every hour. The data scientists have written the following code to read the data for a new key features in the logs. You want to improve the performance of this data read. What should you do?

  1. Specify the TableReference object in the code.
  2. Use .fromQuery operation to read specific fields from the table.
  3. Use of both the Google BigQuery TableSchema and TableFieldSchema classes.
  4. Call a transform that returns TableRow objects, where each element in the PCollection represents a single row in the table.
Show answer and explanation

Correct answer: B. Use .fromQuery operation to read specific fields from the table.

When reading data from BigQuery in Dataflow, using .fromQuery operation to read specific fields from the table is the most effective way to improve performance for large, exponentially growing datasets. This approach pushes down the column selection to BigQuery itself, reducing the amount of data transferred over the network and processed by Dataflow. By querying only the needed fields for key features rather than reading entire rows, bandwidth consumption and processing time are minimized, which is critical when data is growing exponentially. TableReference specifies which table to read but doesn't optimize field selection, TableSchema/TableFieldSchema define structure but don't optimize reads, and returning TableRow objects doesn't inherently improve performance compared to selective field queries.

Why the other options are wrong

  • A. Specifying TableReference only identifies the source table but does not reduce the volume of data being read or transferred.
  • C. Using TableSchema and TableFieldSchema classes defines the data structure but does not optimize the actual read operation or limit fields retrieved.
  • D. Returning TableRow objects for each row does not improve performance; it actually may worsen it by reading complete rows when only specific fields are needed.

Question 44

You have a data stored in BigQuery. The data in the BigQuery dataset must be highly available. You need to define a storage, backup, and recovery strategy of this data that minimizes cost. How should you configure the BigQuery table that have a recovery point objective (RPO) of 30 days?

  1. Set the BigQuery dataset to be regional. In the event of an emergency, use a point-in-time snapshot to recover the data.
  2. Set the BigQuery dataset to be regional. Create a scheduled query to make copies of the data to tables suffixed with the time of the backup. In the event of an emergency, use the backup copy of the table.
  3. Set the BigQuery dataset to be multi-regional. In the event of an emergency, use a point-in-time snapshot to recover the data.
  4. Set the BigQuery dataset to be multi-regional. Create a scheduled query to make copies of the data to tables suffixed with the time of the backup. In the event of an emergency, use the backup copy of the table.
Show answer and explanation

Correct answer: C. Set the BigQuery dataset to be multi-regional. In the event of an emergency, use a point-in-time snapshot to recover the data.

A 30-day RPO requirement means data can be recovered from a point in time within the last 30 days. BigQuery's point-in-time snapshots feature maintains table history for 7 days by default and can be extended to 30 days, allowing recovery from any point within that window. Setting the dataset to multi-regional ensures high availability and protection against regional outages, meeting the high availability requirement while minimizing cost compared to manual backup strategies. Snapshots are a built-in, cost-effective feature that requires no additional infrastructure.

Why the other options are wrong

  • A. Regional datasets are not highly available in the face of regional outages, failing the high availability requirement.
  • B. Creating scheduled backup copies via queries incurs additional storage and query costs compared to using built-in snapshots, and is more operationally complex.
  • D. While multi-regional provides high availability, manually scheduling backup copies is more expensive and complex than using native point-in-time snapshots.

Question 45

You launched a new gaming app almost three years ago. You have been uploading log files from the previous day to a separate Google BigQuery table with the table name format LOGS_yyyymmdd. You have been using table wildcard functions to generate daily and monthly reports for all time ranges. Recently, you discovered that some queries that cover long date ranges are exceeding the limit of 1,000 tables and failing. How can you resolve this issue?

  1. Convert all daily log tables into date-partitioned tables
  2. Convert the sharded tables into a single partitioned table
  3. Enable query caching so you can cache data from previous months
  4. Create separate views to cover each month, and query from these views
Show answer and explanation

Correct answer: B. Convert the sharded tables into a single partitioned table

Converting sharded tables (multiple tables with a naming pattern like LOGS_yyyymmdd) into a single partitioned table eliminates the 1,000-table wildcard query limit. A partitioned table stores all data in one logical table with partition pruning, allowing efficient queries across any time range without encountering table count limitations. This solution directly addresses the root cause of the failure.

Why the other options are wrong

  • A. Converting daily tables into date-partitioned tables still leaves them as separate tables rather than consolidating into a single partitioned table.
  • C. Query caching does not solve the underlying problem of exceeding the 1,000-table limit; cached queries still count against the table limit.
  • D. Creating views for each month only adds another layer of abstraction and does not resolve the fundamental 1,000-table limit on wildcard queries.

Question 46

You work for a car manufacturer and have set up a data pipeline using Google Cloud Pub/Sub to capture anomalous sensor events. You are using a push subscription in Cloud Pub/Sub that calls a custom HTTPS endpoint that you have created to take action of these anomalous events as they occur. Your custom HTTPS endpoint keeps getting an inordinate amount of duplicate messages. What is the most likely cause of these duplicate messages?

  1. The message body for the sensor event is too large.
  2. Your custom endpoint has an out-of-date SSL certificate.
  3. The Cloud Pub/Sub topic has too many messages published to it.
  4. Your custom endpoint is not acknowledging messages within the acknowledgement deadline.
Show answer and explanation

Correct answer: D. Your custom endpoint is not acknowledging messages within the acknowledgement deadline.

When a push subscription endpoint fails to acknowledge messages within the acknowledgment deadline, Pub/Sub automatically retries delivery, resulting in duplicate messages. If the custom HTTPS endpoint does not send a successful acknowledgment response within the configured deadline, Pub/Sub treats the delivery as failed and redelivers the same message.

Why the other options are wrong

  • A. Message body size affects quotas and performance but does not cause duplicate deliveries in push subscriptions.
  • B. An out-of-date SSL certificate would cause connection failures, not duplicate messages; it would prevent delivery entirely.
  • C. High message volume does not inherently cause duplicate messages; Pub/Sub is designed to handle large throughput.

Question 47

You are designing storage for two relational tables that are part of a 10-TB database on Google Cloud. You want to support transactions that scale horizontally. You also want to optimize data for range queries on non-key columns. What should you do?

  1. Use Cloud SQL for storage. Add secondary indexes to support query patterns.
  2. Use Cloud SQL for storage. Use Cloud Dataflow to transform data to support query patterns.
  3. Use Cloud Spanner for storage. Add secondary indexes to support query patterns.
  4. Use Cloud Spanner for storage. Use Cloud Dataflow to transform data to support query patterns.
Show answer and explanation

Correct answer: C. Use Cloud Spanner for storage. Add secondary indexes to support query patterns.

Cloud Spanner is purpose-built for horizontally scalable transactions and supports secondary indexes on non-key columns for optimized range queries. It provides strong consistency, automatic sharding, and the ability to scale transaction throughput across multiple servers. Secondary indexes in Cloud Spanner directly support the range query optimization requirement without requiring data transformation.

Why the other options are wrong

  • A. Cloud SQL does not scale transactions horizontally; it's limited to vertical scaling and is unsuitable for large distributed workloads.
  • B. Cloud SQL combined with Dataflow adds unnecessary complexity; Cloud SQL cannot horizontally scale transactions regardless of data transformation.
  • D. While Cloud Spanner is correct, using Cloud Dataflow to transform data is unnecessary since Cloud Spanner supports secondary indexes natively for range query optimization.

Question 48

Your company is in the process of migrating its on-premises data warehousing solutions to BigQuery. The existing data warehouse uses triggerbased change data capture (CDC) to apply updates from multiple transactional database sources on a daily basis. With BigQuery, your company hopes to improve its handling of CDC so that changes to the source systems are available to query in BigQuery in near-real time using log-based CDC streams, while also optimizing for the performance of applying changes to the data warehouse. Which two steps should they take to ensure that changes are available in the BigQuery reporting table with minimal latency while reducing compute overhead? (Choose two.)

  1. Perform a DML INSERT, UPDATE, or DELETE to replicate each individual CDC record in real time directly on the reporting table.
  2. Insert each new CDC record and corresponding operation type to a staging table in real time.
  3. Periodically DELETE outdated records from the reporting table.
  4. Periodically use a DML MERGE to perform several DML INSERT, UPDATE, and DELETE operations at the same time on the reporting table.
  5. Insert each new CDC record and corresponding operation type in real time to the reporting table, and use a materialized view to expose only the newest version of each unique record.
Show answer and explanation

Correct answer: B, D. Insert each new CDC record and corresponding operation type to a staging table in real time; Periodically use a DML MERGE to perform several DML INSERT, UPDATE, and DELETE operations at the same time on the reporting table.

The optimal approach inserts each CDC record with its operation type to a staging table in real time (B), then periodically uses a MERGE statement (D) to batch-apply multiple insert, update, and delete operations together on the reporting table. This minimizes latency by processing CDC records immediately while optimizing compute by batching DML operations rather than applying each change individually, reducing the overhead of separate DML statements.

Why the other options are wrong

  • A. Performing individual DML operations for each CDC record in real time incurs high compute overhead and is inefficient at scale.
  • C. Periodically deleting outdated records is inefficient; MERGE handles this more elegantly by performing all operations atomically.
  • E. Using a materialized view adds complexity and latency; the MERGE approach directly maintains the reporting table more efficiently.

Question 49

You are designing storage for 20 TB of text files as part of deploying a data pipeline on Google Cloud. Your input data is in CSV format. You want to minimize the cost of querying aggregate values for multiple users who will query the data in Cloud Storage with multiple engines. Which storage service and schema design should you use?

  1. Use Cloud Bigtable for storage. Install the HBase shell on a Compute Engine instance to query the Cloud Bigtable data.
  2. Use Cloud Bigtable for storage. Link as permanent tables in BigQuery for query.
  3. Use Cloud Storage for storage. Link as permanent tables in BigQuery for query.
  4. Use Cloud Storage for storage. Link as temporary tables in BigQuery for query.
Show answer and explanation

Correct answer: C. Use Cloud Storage for storage. Link as permanent tables in BigQuery for query.

For 20 TB of CSV files where multiple users query aggregates using multiple engines, Cloud Storage with permanent BigQuery linked tables minimizes cost and maximizes flexibility. Cloud Storage is the cheapest storage layer, and permanent BigQuery tables provide a unified schema allowing multiple query engines (BigQuery, Dataflow, Spark) to access the data efficiently. Permanent tables enable consistent metadata and schema management across multiple users.

Why the other options are wrong

  • A. Cloud Bigtable is optimized for key-value lookups and real-time access, not for aggregate analytical queries on structured CSV data.
  • B. Cloud Bigtable is unsuitable for CSV data and aggregate queries; it's designed for different access patterns than analytical aggregation.
  • D. Temporary tables require recreation on each query and don't provide persistent metadata, increasing cost and complexity for multi-user scenarios.

Question 50

You are creating a new pipeline in Google Cloud to stream IoT data from Cloud Pub/Sub through Cloud Dataflow to BigQuery. While previewing the data, you notice that roughly 2% of the data appears to be corrupt. You need to modify the Cloud Dataflow pipeline to filter out this corrupt data. What should you do?

  1. Add a SideInput that returns a Boolean if the element is corrupt.
  2. Add a ParDo transform in Cloud Dataflow to discard corrupt elements.
  3. Add a Partition transform in Cloud Dataflow to separate valid data from corrupt data.
  4. Add a GroupByKey transform in Cloud Dataflow to group all of the valid data together and discard the rest.
Show answer and explanation

Correct answer: B. Add a ParDo transform in Cloud Dataflow to discard corrupt elements.

ParDo (Parallel Do) is the standard Cloud Dataflow transform for element-wise processing that can filter, transform, or discard individual elements. It's the appropriate choice for inspecting each data element and removing corrupt records based on validation logic. This is a per-element operation that scales efficiently across the pipeline.

Why the other options are wrong

  • A. SideInputs provide additional context to transforms but are not used for filtering logic; they supply supplementary data rather than validation criteria.
  • C. Partition separates data into multiple outputs but doesn't discard data; it would keep corrupt elements in one partition rather than removing them.
  • D. GroupByKey is for aggregating elements by key and doesn't support element-level filtering or discarding corrupt data.

50 questions in. Want the rest? The first half of the pack, free by email · The full pack

Question 51

You are building a data pipeline on Google Cloud. You need to prepare data using a casual method for a machine-learning process. You want to support a logistic regression model. You also need to monitor and adjust for null values, which must remain real-valued and cannot be removed. What should you do?

  1. Use Cloud Dataprep to find null values in sample source data. Convert all nulls to 'none' using a Cloud Dataproc job.
  2. Use Cloud Dataprep to find null values in sample source data. Convert all nulls to 0 using a Cloud Dataprep job.
  3. Use Cloud Dataflow to find null values in sample source data. Convert all nulls to 'none' using a Cloud Dataprep job.
  4. Use Cloud Dataflow to find null values in sample source data. Convert all nulls to 0 using a custom script.
Show answer and explanation

Correct answer: B. Use Cloud Dataprep to find null values in sample source data. Convert all nulls to 0 using a Cloud Dataprep job.

Cloud Dataprep is designed for interactive data exploration and cleaning, making it ideal for finding and handling null values in sample data. For logistic regression with real-valued features, null values should be converted to 0 (a neutral numeric value) rather than the string 'none', which cannot be used in numeric models. Dataprep can perform this transformation directly without requiring additional jobs.

Why the other options are wrong

  • A. Converting nulls to the string 'none' creates non-numeric values incompatible with logistic regression, which requires real-valued inputs.
  • C. Using Dataflow for null detection is unnecessarily complex; Dataprep is the appropriate exploratory tool, and 'none' is still the wrong conversion value.
  • D. Dataflow with a custom script adds unnecessary complexity when Dataprep can handle the task directly with built-in transformations.

Question 52

You work for a shipping company that uses handheld scanners to read shipping labels. Your company has strict data privacy standards that require scanners to only transmit tracking numbers when events are sent to Kafka topics. A recent software update caused the scanners to accidentally transmit recipients' personally identifiable information (PII) to analytics systems, which violates user privacy rules. You want to quickly build a scalable solution using cloud-native managed services to prevent exposure of PII to the analytics systems. What should you do?

  1. Create an authorized view in BigQuery to restrict access to tables with sensitive data.
  2. Install a third-party data validation tool on Compute Engine virtual machines to check the incoming data for sensitive information.
  3. Use Cloud Logging to analyze the data passed through the total pipeline to identify transactions that may contain sensitive information.
  4. Build a Cloud Function that reads the topics and makes a call to the Cloud Data Loss Prevention (Cloud DLP) API. Use the tagging and confidence levels to either pass or quarantine the data in a bucket for review.
Show answer and explanation

Correct answer: D. Build a Cloud Function that reads the topics and makes a call to the Cloud Data Loss Prevention (Cloud DLP) API. Use the tagging and confidence levels to either pass or quarantine the data in a bucket for review.

Cloud Data Loss Prevention (DLP) API is Google's managed service specifically designed to detect, classify, and protect sensitive data including PII. Using a Cloud Function to read from Kafka topics and call the DLP API provides a scalable, serverless solution that inspects incoming data before it reaches analytics systems. The DLP API's tagging and confidence levels allow flexible policies to quarantine suspicious data for review while allowing legitimate tracking numbers through, directly addressing the PII exposure problem at the data ingestion boundary with minimal operational overhead.

Why the other options are wrong

  • A. BigQuery authorized views restrict access to data after it reaches the data warehouse, but do not prevent PII from being transmitted to analytics systems in the first place.
  • B. Installing validation tools on Compute Engine VMs is not cloud-native, not managed, and does not scale automatically with varying data volumes.
  • C. Cloud Logging analyzes logs after data has already been processed and transmitted; it does not prevent exposure and is reactive rather than preventive.

Question 53

You currently have a single on-premises Kafka cluster in a data center in the us-east region that is responsible for ingesting messages from IoT devices globally. Because large parts of globe have poor internet connectivity, messages sometimes batch at the edge, come in all at once, and cause a spike in load on your Kafka cluster. This is becoming difficult to manage and prohibitively expensive. What is the Google-recommended cloud native architecture for this scenario?

  1. Edge TPUs as sensor devices for storing and transmitting the messages.
  2. Cloud Dataflow connected to the Kafka cluster to scale the processing of incoming messages.
  3. An IoT gateway connected to Cloud Pub/Sub, with Cloud Dataflow to read and process the messages from Cloud Pub/Sub.
  4. A Kafka cluster virtualized on Compute Engine in us-east with Cloud Load Balancing to connect to the devices around the world.
Show answer and explanation

Correct answer: C. An IoT gateway connected to Cloud Pub/Sub, with Cloud Dataflow to read and process the messages from Cloud Pub/Sub.

Google's recommended cloud-native architecture for globally distributed IoT devices with intermittent connectivity involves an IoT gateway that buffers and smooths incoming messages, feeding them into Cloud Pub/Sub for reliable ingestion at scale. Cloud Dataflow then processes these messages from Pub/Sub, automatically scaling to handle burst traffic. This architecture decouples message ingestion from processing, handles connectivity issues gracefully, and provides elastic scaling without the management burden of an on-premises Kafka cluster.

Why the other options are wrong

  • A. Edge TPUs are specialized hardware for machine learning inference, not for message storage and transmission.
  • B. Connecting Dataflow directly to an on-premises Kafka cluster does not solve the edge connectivity problem or eliminate the need to manage on-premises infrastructure.
  • D. Virtualizing Kafka on Compute Engine still requires on-premises management, does not improve edge connectivity, and does not leverage Google Cloud's native services.

Question 54

You want to analyze hundreds of thousands of social media posts daily at the lowest cost and with the fewest steps. You have the following requirements: • You will batch-load the posts once per day and run them through the Cloud Natural Language API. • You will extract topics and sentiment from the posts. • You must store the raw posts for archiving and reprocessing. • You will create dashboards to be shared with people both inside and outside your organization. You need to store both the data extracted from the API to perform analysis as well as the raw social media posts for historical archiving. What should you do?

  1. Store the social media posts and the data extracted from the API in BigQuery.
  2. Store the social media posts and the data extracted from the API in Cloud SQL.
  3. Store the raw social media posts in Cloud Storage, and write the data extracted from the API into BigQuery.
  4. Feed to social media posts into the API directly from the source, and write the extracted data from the API into BigQuery.
Show answer and explanation

Correct answer: C. Store the raw social media posts in Cloud Storage, and write the data extracted from the API into BigQuery.

This solution optimizes cost and workflow: raw social media posts are archived in Cloud Storage (cost-effective for large volumes of unstructured data), while extracted API results (topics, sentiment) are stored in BigQuery for efficient analysis and dashboard creation. BigQuery enables querying, aggregation, and sharing with internal and external users through built-in visualization and access controls.

Why the other options are wrong

  • A. Storing raw posts in BigQuery is expensive for archival purposes and wastes resources since raw posts won't be queried frequently.
  • B. Cloud SQL is designed for transactional workloads and scales poorly for hundreds of thousands of daily records and dashboard queries.
  • D. Feeding posts directly from the source to the API eliminates the ability to reprocess historical data, violating the archiving requirement.

Question 55

Your software uses a simple JSON format for all messages. These messages are published to Google Cloud Pub/Sub, then processed with Google Cloud Dataflow to create a real-time dashboard for the CFO. During testing, you notice that some messages are missing in the dashboard. You check the logs, and all messages are being published to Cloud Pub/Sub successfully. What should you do next?

  1. Check the dashboard application to see if it is not displaying correctly.
  2. Run a fixed dataset through the Cloud Dataflow pipeline and analyze the output.
  3. Use Google Stackdriver Monitoring on Cloud Pub/Sub to find the missing messages.
  4. Switch Cloud Dataflow to pull messages from Cloud Pub/Sub instead of Cloud Pub/Sub pushing messages to Cloud Dataflow.
Show answer and explanation

Correct answer: B. Run a fixed dataset through the Cloud Dataflow pipeline and analyze the output.

Running a fixed dataset through the Cloud Dataflow pipeline and analyzing the output is the appropriate diagnostic step. Since messages are confirmed to reach Pub/Sub, the issue must be in the pipeline processing logic or transformation. Testing with a controlled dataset allows isolation of the problem, whether messages are being filtered out, transformed unexpectedly, or lost due to pipeline logic errors, without the complexity of real-time data flow.

Why the other options are wrong

  • A. If the dashboard application was at fault, the issue would appear differently; we need to confirm data is actually reaching the dashboard from the pipeline first.
  • C. Stackdriver Monitoring on Pub/Sub would only confirm message delivery to Pub/Sub (already verified), not whether the pipeline is processing them correctly.
  • D. Switching from push to pull mode doesn't address the root cause of missing messages and adds unnecessary complexity; the issue is in pipeline processing, not the delivery mechanism.

Question 56

You are a retailer that wants to integrate your online sales capabilities with different in-home assistants, such as Google Home. You need to interpret customer voice commands and issue an order to the backend systems. Which solutions should you choose?

  1. Speech-to-Text API
  2. Cloud Natural Language API
  3. Dialogflow Enterprise Edition
  4. AutoML Natural Language
Show answer and explanation

Correct answer: C. Dialogflow Enterprise Edition

Dialogflow Enterprise Edition is purpose-built for creating conversational interfaces and voice-enabled applications. It integrates with Google Home and other assistants, understands natural language commands, manages conversation flow, and can trigger backend actions like issuing orders. It handles the full conversational AI pipeline from speech recognition through intent understanding to action execution.

Why the other options are wrong

  • A. Speech-to-Text API only converts audio to text; it doesn't interpret commands or manage the conversation flow needed for order placement.
  • B. Cloud Natural Language API performs text analysis but doesn't handle voice input, conversation management, or integration with in-home assistants.
  • D. AutoML Natural Language is for training custom text classification models, not for building conversational interfaces or voice command systems.

Question 57

After migrating ETL jobs to run on BigQuery, you need to verify that the output of the migrated jobs is the same as the output of the original. You've loaded a table containing the output of the original job and want to compare the contents with output from the migrated job to show that they are identical. The tables do not contain a primary key column that would enable you to join them together for comparison. What should you do?

  1. Select random samples from the tables using the RAND() function and compare the samples.
  2. Select random samples from the tables using the HASH() function and compare the samples.
  3. Use a Dataproc cluster and the BigQuery Hadoop connector to read the data from each table and calculate a hash from non-timestamp columns of the table after sorting. Compare the hashes of each table.
  4. Create stratified random samples using the OVER() function and compare equivalent samples from each table.
Show answer and explanation

Correct answer: C. Use a Dataproc cluster and the BigQuery Hadoop connector to read the data from each table and calculate a hash from non-timestamp columns of the table after sorting. Compare the hashes of each table.

Using a Dataproc cluster with the BigQuery Hadoop connector to calculate hashes from sorted non-timestamp columns is the most reliable method for verifying table equivalence without primary keys. This approach sorts both tables identically, excludes timestamp columns that may differ due to processing time, and generates a single hash for each table that can be compared. This method is deterministic and captures the actual data content differences while ignoring non-critical timing variations that occur between original and migrated jobs.

Why the other options are wrong

  • A. Random sampling cannot guarantee that all data is identical; it only provides probabilistic verification.
  • B. Using HASH() on random samples within SQL does not provide comprehensive verification of table equivalence.
  • D. Stratified random sampling still only provides probabilistic verification rather than definitive proof of equivalence.

Question 58

You are building a model to make clothing recommendations. You know a user's fashion preference is likely to change over time, so you build a data pipeline to stream new data back to the model as it becomes available. How should you use this data to train the model?

  1. Continuously retrain the model on just the new data.
  2. Continuously retrain the model on a combination of existing data and the new data.
  3. Train on the existing data while using the new data as your test set.
  4. Train on the new data while using the existing data as your test set.
Show answer and explanation

Correct answer: B. Continuously retrain the model on a combination of existing data and the new data.

When continuously retraining a model on streaming data where user preferences evolve, you must balance adaptation to new trends with retention of historical patterns. Retraining on only new data causes catastrophic forgetting where the model loses previously learned knowledge. Combining existing data with new data maintains stability while allowing the model to learn evolving preferences. Options C and D incorrectly use new data as a test set, which violates the principle that test data should be independent. Option A causes the model to forget previously valid patterns.

Why the other options are wrong

  • A. Training only on new data causes catastrophic forgetting of previously learned valid patterns.
  • C. Using new data as a test set violates proper train-test separation and doesn't retrain the model on it.
  • D. Using existing data as a test set violates proper train-test separation and prevents learning from new data.

Question 59

A data scientist has created a BigQuery ML model and asks you to create an ML pipeline to serve predictions. You have a REST API application with the requirement to serve predictions for an individual user ID with latency under 100 milliseconds. You use the following query to generate predictions: SELECT predicted_label, user_id FROM ML.PREDICT (MODEL 'dataset.model', table user_features). How should you create the ML pipeline?

  1. Add a WHERE clause to the query, and grant the BigQuery Data Viewer role to the application service account.
  2. Create an Authorized View with the provided query. Share the dataset that contains the view with the application service account.
  3. Create a Dataflow pipeline using BigQueryIO to read results from the query. Grant the Dataflow Worker role to the application service account.
  4. Create a Dataflow pipeline using BigQueryIO to read predictions for all users from the query. Write the results to Bigtable using BigtableIO. Grant the Bigtable Reader role to the application service account so that the application can read predictions for individual users from Bigtable.
Show answer and explanation

Correct answer: D. Create a Dataflow pipeline using BigQueryIO to read predictions for all users from the query. Write the results to Bigtable using BigtableIO. Grant the Bigtable Reader role to the application service account so that the application can read predictions for individual users from Bigtable.

To achieve sub-100 millisecond latency for individual user predictions, you must pre-compute all predictions and store them in a low-latency database. Dataflow reads all predictions from BigQuery ML, writes results to Bigtable (which provides fast key-value lookups), and the application queries Bigtable by user ID to retrieve predictions. This approach avoids querying BigQuery for each request, which would exceed latency requirements.

Why the other options are wrong

  • A. Querying BigQuery with WHERE clauses for individual users cannot meet the 100ms latency requirement.
  • B. Authorized Views still require querying BigQuery per request, which exceeds 100ms latency for individual predictions.
  • C. Reading from BigQuery on each API request would exceed 100ms latency; Dataflow is meant for batch processing, not low-latency serving.

Question 60

You work for a manufacturing company that sources up to 750 different components, each from a different supplier. You've collected a labeled dataset that has on average 1000 examples for each unique component. Your team wants to implement an app to help warehouse workers recognize incoming components based on a photo of the component. You want to implement the first working version of this app (as Proof-OfConcept) within a few working days. What should you do?

  1. Use Cloud Vision AutoML with the existing dataset.
  2. Use Cloud Vision AutoML, but reduce your dataset twice.
  3. Use Cloud Vision API by providing custom labels as recognition hints.
  4. Train your own image recognition model leveraging transfer learning techniques.
Show answer and explanation

Correct answer: A. Use Cloud Vision AutoML with the existing dataset.

Cloud Vision AutoML is purpose-built for rapid prototyping of image classification models without requiring machine learning expertise. With 750 classes and 1000 examples per class, you have a well-labeled dataset that meets AutoML's requirements. AutoML handles the complexity of model training, hyperparameter tuning, and deployment automatically, enabling a working proof-of-concept within days. The dataset size is appropriate for AutoML without reduction.

Why the other options are wrong

  • B. Reducing the dataset would degrade model accuracy unnecessarily when the current size is suitable for AutoML.
  • C. Cloud Vision API's custom labels are for object detection hints, not a full classification solution, and wouldn't meet the PoC requirement.
  • D. Building a custom transfer learning model requires more expertise and time than needed for a quick PoC when AutoML is available.

Question 61

Your company produces 20,000 files every hour. Each data file is formatted as a comma separated values (CSV) file that is less than 4 KB. All files must be ingested on Google Cloud Platform before they can be processed. Your company site has a 200 ms latency to Google Cloud, and your Internet connection bandwidth is limited as 50 Mbps. You currently deploy a secure FTP (SFTP) server on a virtual machine in Google Compute Engine as the data ingestion point. A local SFTP client runs on a dedicated machine to transmit the CSV files as is. The goal is to make reports with data from the previous day available to the executives by 10:00 a.m. each day. This design is barely able to keep up with the current volume, even though the bandwidth utilization is rather low. You are told that due to seasonality, your company expects the number of files to double for the next three months. Which two actions should you take? (Choose two.)

  1. Introduce data compression for each file to increase the rate file of file transfer.
  2. Contact your internet service provider (ISP) to increase your maximum bandwidth to at least 100 Mbps.
  3. Redesign the data ingestion process to use gsutil tool to send the CSV files to a storage bucket in parallel.
  4. Assemble 1,000 files into a tape archive (TAR) file. Transmit the TAR files instead, and disassemble the CSV files in the cloud upon receiving them.
  5. Create an S3-compatible storage endpoint in your network, and use Google Cloud Storage Transfer Service to transfer on-premises data to the designated storage bucket.
Show answer and explanation

Correct answer: C, D. Redesign the data ingestion process to use gsutil tool to send the CSV files to a storage bucket in parallel; Assemble 1,000 files into a tape archive (TAR) file. Transmit the TAR files instead, and disassemble the CSV files in the cloud upon receiving them.

Option C redesigns ingestion to use gsutil for parallel uploads to Cloud Storage, which is far more efficient than sequential SFTP transfers and leverages Google Cloud's native tooling. Parallel uploads overcome the latency bottleneck by multiplexing connections. Option D assembles files into TAR archives to reduce the number of network round-trips, dramatically improving throughput efficiency since fewer connections and handshakes are required for the same data volume. These two approaches address the fundamental issue: the design relies on sequential single-connection transfers over limited bandwidth. Option A (compression) won't help much since bandwidth utilization is low, indicating the bottleneck is latency, not bandwidth saturation. Option B (increasing ISP bandwidth) doesn't solve the latency problem. Option E uses a third-party transfer service that doesn't address the real problem of sequential transfer inefficiency.

Why the other options are wrong

  • A. Compression doesn't address the core problem; bandwidth utilization is already low, meaning latency and connection overhead are the real bottlenecks.
  • B. Bandwidth is not the constraint; latency and sequential transfer inefficiency are the issues, so increasing bandwidth provides minimal benefit.
  • E. Cloud Storage Transfer Service is designed for cloud-to-cloud or large-scale transfers, not for this on-premises scenario, and doesn't solve the latency and sequential transfer problems.

Question 62

You are designing a data processing pipeline. The pipeline must be able to scale automatically as load increases. Messages must be processed at least once and must be ordered within windows of 1 hour. How should you design the solution?

  1. Use Apache Kafka for message ingestion and use Cloud Dataproc for streaming analysis.
  2. Use Apache Kafka for message ingestion and use Cloud Dataflow for streaming analysis.
  3. Use Cloud Pub/Sub for message ingestion and Cloud Dataproc for streaming analysis.
  4. Use Cloud Pub/Sub for message ingestion and Cloud Dataflow for streaming analysis.
Show answer and explanation

Correct answer: D. Use Cloud Pub/Sub for message ingestion and Cloud Dataflow for streaming analysis.

Cloud Pub/Sub provides reliable message ingestion with automatic scaling and at-least-once delivery guarantees, while Cloud Dataflow provides streaming analysis with native windowing support (including 1-hour windows) and automatic horizontal scaling. Dataflow's streaming engine handles ordered processing within windows and manages state efficiently, making this the optimal combination for the stated requirements.

Why the other options are wrong

  • A. Apache Kafka requires manual cluster management and doesn't auto-scale; Cloud Dataproc is batch-focused and not ideal for real-time streaming with ordered windowing.
  • B. While Kafka can buffer messages, it requires operational overhead to manage; Cloud Dataflow is the better choice for the streaming analysis component.
  • C. Cloud Pub/Sub is correct, but Cloud Dataproc is not designed for low-latency streaming analysis with sophisticated windowing and ordering guarantees.

Question 63

You operate a logistics company, and you want to improve event delivery reliability for vehicle-based sensors. You operate small data centers around the world to capture these events, but leased lines that provide connectivity from your event collection infrastructure to your event processing infrastructure are unreliable, with unpredictable latency. You want to address this issue in the most costeffective way. What should you do?

  1. Deploy small Kafka clusters in your data centers to buffer events.
  2. Have the data acquisition devices publish data to Cloud Pub/Sub.
  3. Establish a Cloud Interconnect between all remote data centers and Google.
  4. Write a Cloud Dataflow pipeline that aggregates all data in session windows.
Show answer and explanation

Correct answer: B. Have the data acquisition devices publish data to Cloud Pub/Sub.

Cloud Pub/Sub is designed to handle unreliable network conditions and unpredictable latency. It provides built-in buffering, automatic retries, and guaranteed at-least-once delivery semantics without requiring the company to manage infrastructure. This is cost-effective compared to building local Kafka clusters or establishing dedicated interconnects, and it abstracts away network reliability concerns.

Why the other options are wrong

  • A. While Kafka provides buffering, deploying and managing Kafka clusters across multiple data centers adds operational overhead and cost.
  • C. Cloud Interconnect is expensive and designed for guaranteed high-performance connectivity, not for handling unreliable networks cost-effectively.
  • D. Session windows in Dataflow are for time-based aggregation within the pipeline, not for addressing upstream network reliability or buffering issues.

Question 64

You are implementing security best practices on your data pipeline. Currently, you are manually executing jobs as the Project Owner. You want to automate these jobs by taking nightly batch files containing non-public information from Google Cloud Storage, processing them with a Spark Scala job on a Google Cloud Dataproc cluster, and depositing the results into Google BigQuery. How should you securely run this workload?

  1. Restrict the Google Cloud Storage bucket so only you can see the files
  2. Grant the Project Owner role to a service account, and run the job with it
  3. Use a service account with the ability to read the batch files and to write to BigQuery
  4. Use a user account with the Project Viewer role on the Cloud Dataproc cluster to read the batch files and write to BigQuery
Show answer and explanation

Correct answer: C. Use a service account with the ability to read the batch files and to write to BigQuery

The correct approach to securely run this workload is to use a service account with minimal, specific permissions, read access to the Cloud Storage bucket and write access to BigQuery. This follows the principle of least privilege, which is a security best practice. Service accounts are designed for machine-to-machine authentication and should be used for automated workloads rather than user accounts or overly permissive roles.

Why the other options are wrong

  • A. Restricting visibility to only the Project Owner does not address the automation requirement and does not establish a secure service account for the job.
  • B. Granting the Project Owner role to a service account violates the principle of least privilege by giving excessive permissions beyond what is needed for this specific task.
  • D. Using a user account is not appropriate for automated jobs; service accounts are the correct choice for unattended workloads, and Project Viewer role is insufficient for writing to BigQuery.

Question 65

You are operating a Cloud Dataflow streaming pipeline. The pipeline aggregates events from a Cloud Pub/Sub subscription source, within a window, and sinks the resulting aggregation to a Cloud Storage bucket. The source has consistent throughput. You want to monitor an alert on behavior of the pipeline with Cloud Stackdriver to ensure that it is processing data. Which Stackdriver alerts should you create?

  1. An alert based on a decrease of subscription/num_undelivered_messages for the source and a rate of change increase of instance/storage/ used_bytes for the destination
  2. An alert based on an increase of subscription/num_undelivered_messages for the source and a rate of change decrease of instance/storage/ used_bytes for the destination
  3. An alert based on a decrease of instance/storage/used_bytes for the source and a rate of change increase of subscription/ num_undelivered_messages for the destination
  4. An alert based on an increase of instance/storage/used_bytes for the source and a rate of change decrease of subscription/ num_undelivered_messages for the destination
Show answer and explanation

Correct answer: B. An alert based on an increase of subscription/num_undelivered_messages for the source and a rate of change decrease of instance/storage/ used_bytes for the destination

When a Cloud Dataflow streaming pipeline is processing data normally with consistent source throughput, subscription/num_undelivered_messages should remain stable or low; an increase in this metric indicates the pipeline is not keeping up with ingestion. Simultaneously, instance/storage/used_bytes at the destination should increase as data is successfully written to Cloud Storage; a decrease in the rate of change would signal a processing slowdown. These metrics together indicate healthy data flow through the pipeline.

Why the other options are wrong

  • A. A decrease in undelivered messages does not indicate a problem; this metric is not the right indicator of pipeline health, and used_bytes increasing is expected but not the focus of the alert.
  • C. Mixing source and destination metrics incorrectly; used_bytes is not a source metric, and this assignment reverses the logical indicators of pipeline health.
  • D. Increasing storage metrics at the source does not make sense in this architecture, and decreasing storage rate at the destination suggests data is not being written, indicating failure.

Question 66

Your United States-based company has created an application for assessing and responding to user actions. The primary table's data volume grows by 250,000 records per second. Many third parties use your application's APIs to build the functionality into their own frontend applications. Your application's APIs should comply with the following requirements: • Single global endpoint • ANSI SQL support • Consistent access to the most up-to-date data What should you do?

  1. Implement BigQuery with no region selected for storage or processing.
  2. Implement Cloud Spanner with the leader in North America and read-only replicas in Asia and Europe.
  3. Implement Cloud SQL for PostgreSQL with the master in North America and read replicas in Asia and Europe.
  4. Implement Bigtable with the primary cluster in North America and secondary clusters in Asia and Europe.
Show answer and explanation

Correct answer: B. Implement Cloud Spanner with the leader in North America and read-only replicas in Asia and Europe.

Cloud Spanner meets all requirements: it provides a single global endpoint with ANSI SQL support and strong consistency guarantees for up-to-date data access. With 250,000 records per second throughput, Spanner can handle the write volume. A leader-replica configuration with the leader in North America and read-only replicas in Asia and Europe ensures consistent global access while providing geographic distribution for lower latency.

Why the other options are wrong

  • A. BigQuery without region selection doesn't provide the sub-100ms latency required for API serving and lacks strong consistency guarantees.
  • C. Cloud SQL read replicas provide eventual consistency, not the consistent fresh-data access required; it also cannot handle 250k records/second write throughput.
  • D. Bigtable lacks ANSI SQL support, which is an explicit requirement for the API.

Question 67

You are a head of BI at a large enterprise company with multiple business units that each have different priorities and budgets. You use on-demand pricing for BigQuery with a quota of 2K concurrent on-demand slots per project. Users at your organization sometimes don't get slots to execute their query and you need to correct this. You'd like to avoid introducing new projects to your account. What should you do?

  1. Convert your batch BQ queries into interactive BQ queries.
  2. Create an additional project to overcome the 2K on-demand per-project quota.
  3. Switch to flat-rate pricing and establish a hierarchical priority model for your projects.
  4. Increase the amount of concurrent slots per project at the Quotas page at the Cloud Console.
Show answer and explanation

Correct answer: C. Switch to flat-rate pricing and establish a hierarchical priority model for your projects.

Switching to flat-rate pricing with a hierarchical priority model allows the organization to overcome the 2K on-demand slot limitation while managing costs across multiple business units with different priorities. Flat-rate pricing provides guaranteed capacity regardless of the per-project concurrency limits, and a priority model ensures critical business unit queries execute first. This approach avoids creating additional projects while providing the flexibility needed to serve multiple competing business units effectively.

Why the other options are wrong

  • A. Converting batch queries to interactive queries does not increase available slots and may increase costs.
  • B. The requirement explicitly states avoiding new projects to overcome quota limitations.
  • D. The 2K concurrent slots per-project quota cannot be increased; this is an account-level limit that requires a different pricing model.

Question 68

You need to migrate a 2TB relational database to Google Cloud Platform. You do not have the resources to significantly refactor the application that uses this database and cost to operate is of primary concern. Which service do you select for storing and serving your data?

  1. Cloud Spanner
  2. Cloud Bigtable
  3. Cloud Firestore
  4. Cloud SQL
Show answer and explanation

Correct answer: D. Cloud SQL

Cloud SQL is the correct choice for migrating a relational database when refactoring is minimal and cost is primary. It provides a managed relational database service compatible with MySQL, PostgreSQL, and SQL Server. For a 2TB database with limited refactoring, Cloud SQL offers the lowest operational cost and effort since it requires no application changes, handles backups and maintenance automatically, and avoids the architectural complexity of other options.

Why the other options are wrong

  • A. Cloud Spanner is expensive and designed for globally distributed applications requiring horizontal scaling, not cost-optimized legacy database migration.
  • B. Cloud Bigtable is a NoSQL wide-column store, not suitable for relational databases and would require significant application refactoring.
  • C. Cloud Firestore is a document database requiring application refactoring and is not suitable for complex relational schemas.

Question 69

You decided to use Cloud Datastore to ingest vehicle telemetry data in real time. You want to build a storage system that will account for the longterm data growth, while keeping the costs low. You also want to create snapshots of the data periodically, so that you can make a point-in-time (PIT) recovery, or clone a copy of the data for Cloud Datastore in a different environment. You want to archive these snapshots for a long time. Which two methods can accomplish this? (Choose two.)

  1. Use managed export, and store the data in a Cloud Storage bucket using Nearline or Coldline class.
  2. Use managed export, and then import to Cloud Datastore in a separate project under a unique namespace reserved for that export.
  3. Use managed export, and then import the data into a BigQuery table created just for that export, and delete temporary export files.
  4. Write an application that uses Cloud Datastore client libraries to read all the entities. Treat each entity as a BigQuery table row via BigQuery streaming insert. Assign an export timestamp for each export, and attach it as an extra column for each row. Make sure that the BigQuery table is partitioned using the export timestamp column.
  5. Write an application that uses Cloud Datastore client libraries to read all the entities. Format the exported data into a JSON file. Apply compression before storing the data in Cloud Source Repositories.
Show answer and explanation

Correct answer: A, B. Use managed export, and store the data in a Cloud Storage bucket using Nearline or Coldline class; Use managed export, and then import to Cloud Datastore in a separate project under a unique namespace reserved for that export.

Datastore managed export writes a restorable snapshot, and storing it in a Nearline or Coldline bucket keeps long-term archival cheap while still allowing a point-in-time restore. Importing a managed export into Datastore in a separate project, under its own namespace, is how you clone a copy of the data for another environment. Both methods keep the data in Datastore's native, re-importable format.

Why the other options are wrong

  • C. Loading the export into BigQuery and deleting the export files removes the Datastore-format snapshot, so you can no longer restore or clone into Datastore.
  • D. A custom BigQuery streaming copy is costly, complex and does not produce a snapshot that can be restored into Datastore.
  • E. Cloud Source Repositories is for source code, not for archiving large data snapshots, and a custom JSON dump is not a managed restore path.

Question 70

You work for a large fast food restaurant chain with over 400,000 employees. You store employee information in Google BigQuery in a Users table consisting of a FirstName field and a LastName field. A member of IT is building an application and asks you to modify the schema and data in BigQuery so the application can query a FullName field consisting of the value of the FirstName field concatenated with a space, followed by the value of the LastName field for each employee. How can you make that data available while minimizing cost?

  1. Create a view in BigQuery that concatenates the FirstName and LastName field values to produce the FullName.
  2. Add a new column called FullName to the Users table. Run an UPDATE statement that updates the FullName column for each user with the concatenation of the FirstName and LastName values.
  3. Create a Google Cloud Dataflow job that queries BigQuery for the entire Users table, concatenates the FirstName value and LastName value for each user, and loads the proper values for FirstName, LastName, and FullName into a new table in BigQuery.
  4. Use BigQuery to export the data for the table to a CSV file. Create a Google Cloud Dataproc job to process the CSV file and output a new CSV file containing the proper values for FirstName, LastName and FullName. Run a BigQuery load job to load the new CSV file into BigQuery.
Show answer and explanation

Correct answer: A. Create a view in BigQuery that concatenates the FirstName and LastName field values to produce the FullName.

A BigQuery view provides the most cost-effective solution because views are logical constructs that don't store additional data, they compute the FullName field on-the-fly during query execution. This avoids storage costs and UPDATE statement costs while still making the FullName field available to the application. Option B adds storage overhead and incurs costs for the UPDATE operation. Options C and D both involve unnecessary data movement and processing through Dataflow or Dataproc, which are more expensive than a simple view for this use case.

Why the other options are wrong

  • B. Adding a physical FullName column and running UPDATE statements incurs both storage costs and computation costs for the bulk update operation.
  • C. A Dataflow job introduces unnecessary complexity and computational expense for a simple field concatenation that a view can handle.
  • D. Exporting to CSV, processing through Dataproc, and reloading involves multiple expensive steps and data movement that a view avoids entirely.

Question 71

You are training a spam classifier. You notice that you are overfitting the training data. Which three actions can you take to resolve this problem? (Choose three.)

  1. Get more training examples
  2. Reduce the number of training examples
  3. Use a smaller set of features
  4. Use a larger set of features
  5. Increase the regularization parameters
  6. Decrease the regularization parameters
Show answer and explanation

Correct answer: A, C, E. Get more training examples; Use a smaller set of features; Increase the regularization parameters

Getting more training examples (A) reduces overfitting by providing more diverse data patterns to learn from. Using a smaller set of features (C) reduces model complexity and prevents the model from fitting to noise. Increasing regularization parameters (E) adds penalty terms that constrain the model weights, discouraging overfitting. These three techniques directly address overfitting by either increasing data diversity, reducing model complexity, or adding constraints. Option B reduces data, which worsens overfitting. Option D increases model complexity, increasing overfitting risk. Option F decreases regularization, allowing more overfitting.

Why the other options are wrong

  • B. Reducing training examples exacerbates overfitting by providing less data for the model to learn general patterns.
  • D. Increasing the feature set increases model complexity and the capacity to overfit to training data.
  • F. Decreasing regularization removes constraints on the model, allowing it to overfit more easily.

Question 72

You have historical data covering the last three years in BigQuery and a data pipeline that delivers new data to BigQuery daily. You have noticed that when the Data Science team runs a query filtered on a date column and limited to 30`"90 days of data, the query scans the entire table. You also noticed that your bill is increasing more quickly than you expected. You want to resolve the issue as cost-effectively as possible while maintaining the ability to conduct SQL queries. What should you do?

  1. Re-create the tables using DDL. Partition the tables by a column containing a TIMESTAMP or DATE Type.
  2. Recommend that the Data Science team export the table to a CSV file on Cloud Storage and use Cloud Datalab to explore the data by reading the files directly.
  3. Modify your pipeline to maintain the last - 3090 days of data in one table and the longer history in a different table to minimize full table scans over the entire history.
  4. Write an Apache Beam pipeline that creates a BigQuery table per day. Recommend that the Data Science team use wildcards on the table name suffixes to select the data they need.
Show answer and explanation

Correct answer: A. Re-create the tables using DDL. Partition the tables by a column containing a TIMESTAMP or DATE Type.

Partitioning tables by a TIMESTAMP or DATE column allows BigQuery to prune partitions that fall outside the query filter range, dramatically reducing the amount of data scanned. When the Data Science team queries 30-90 days of data, only those partitions are scanned rather than the entire three-year table, resulting in significant cost savings while maintaining full SQL query capability.

Why the other options are wrong

  • B. Exporting to CSV and using Cloud Datalab eliminates the benefits of BigQuery's query optimization and increases operational complexity.
  • C. Maintaining separate tables adds complexity and doesn't leverage BigQuery's native partitioning capabilities, which are more cost-effective.
  • D. Creating a table per day creates maintenance overhead and requires wildcard queries, which is less efficient than native partitioning and more complex to manage.

Question 73

You need to copy millions of sensitive patient records from a relational database to BigQuery. The total size of the database is 10 TB. You need to design a solution that is secure and time-efficient. What should you do?

  1. Export the records from the database as an Avro file. Upload the file to GCS using gsutil, and then load the Avro file into BigQuery using the BigQuery web UI in the GCP Console.
  2. Export the records from the database as an Avro file. Copy the file onto a Transfer Appliance and send it to Google, and then load the Avro file into BigQuery using the BigQuery web UI in the GCP Console.
  3. Export the records from the database into a CSV file. Create a public URL for the CSV file, and then use Storage Transfer Service to move the file to Cloud Storage. Load the CSV file into BigQuery using the BigQuery web UI in the GCP Console.
  4. Export the records from the database as an Avro file. Create a public URL for the Avro file, and then use Storage Transfer Service to move the file to Cloud Storage. Load the Avro file into BigQuery using the BigQuery web UI in the GCP Console.
Show answer and explanation

Correct answer: A. Export the records from the database as an Avro file. Upload the file to GCS using gsutil, and then load the Avro file into BigQuery using the BigQuery web UI in the GCP Console.

10 TB can be uploaded over the network in a reasonable time, and gsutil transfers data to Cloud Storage over encrypted TLS connections. Avro keeps the schema and loads efficiently into BigQuery. This is both secure and faster than shipping physical hardware.

Why the other options are wrong

  • B. A Transfer Appliance adds days of shipping and handling, which is slower than uploading 10 TB over the network.
  • C. A public URL exposes sensitive patient records to anyone, and CSV loses schema information.
  • D. Publishing sensitive patient data through a public URL is a serious security violation.

Question 74

Your company receives both batch- and stream-based event data. You want to process the data using Google Cloud Dataflow over a predictable time period. However, you realize that in some instances data can arrive late or out of order. How should you design your Cloud Dataflow pipeline to handle data that is late or out of order?

  1. Set a single global window to capture all the data.
  2. Set sliding windows to capture all the lagged data.
  3. Use watermarks and timestamps to capture the lagged data.
  4. Ensure every datasource type (stream or batch) has a timestamp, and use the timestamps to define the logic for lagged data.
Show answer and explanation

Correct answer: C. Use watermarks and timestamps to capture the lagged data.

Watermarks and timestamps are the canonical Dataflow mechanisms for handling late or out-of-order data. Watermarks track progress through event time and allow the pipeline to emit results when data arrives late, while timestamps define the event time for each element. This combination enables proper handling of both early and late data in streaming pipelines.

Why the other options are wrong

  • A. A single global window captures all data but provides no mechanism to distinguish or handle late and out-of-order elements differently.
  • B. Sliding windows alone don't address late or out-of-order data; watermarks are needed to properly handle arrival delays.
  • D. Simply ensuring timestamps exist doesn't provide the windowing and watermarking logic necessary to handle late data correctly.

Question 75

You are working on a sensitive project involving private user data. You have set up a project on Google Cloud Platform to house your work internally. An external consultant is going to assist with coding a complex transformation in a Google Cloud Dataflow pipeline for your project. How should you maintain users' privacy?

  1. Grant the consultant the Viewer role on the project.
  2. Grant the consultant the Cloud Dataflow Developer role on the project.
  3. Create a service account and allow the consultant to log on with it.
  4. Create an anonymized sample of the data for the consultant to work with in a different project.
Show answer and explanation

Correct answer: D. Create an anonymized sample of the data for the consultant to work with in a different project.

When an external consultant needs to work with sensitive user data, the best practice for maintaining privacy is to provide them with an anonymized sample dataset in a separate project. This approach allows the consultant to develop and test the Dataflow transformation without exposure to actual private user data, minimizing privacy risk while still enabling them to complete their work effectively. The anonymized sample should contain the same structure and characteristics as the production data but with personally identifiable information removed or obfuscated.

Why the other options are wrong

  • A. The Viewer role grants read access to all project resources, including sensitive data, which is unnecessary and violates privacy principles.
  • B. The Cloud Dataflow Developer role allows deploying and managing pipelines with access to actual data, unnecessarily exposing the consultant to private user information.
  • C. Creating a service account for the consultant to use still grants access to the actual sensitive data in the project, failing to protect user privacy.

75 questions in. Want the rest? The first half of the pack, free by email · The full pack

Question 76

Your organization has been collecting and analyzing data in Google BigQuery for 6 months. The majority of the data analyzed is placed in a timepartitioned table named events_partitioned. To reduce the cost of queries, your organization created a view called events, which queries only the last 14 days of data. The view is described in legacy SQL. Next month, existing applications will be connecting to BigQuery to read the events data via an ODBC connection. You need to ensure the applications can connect. Which two actions should you take? (Choose two.)

  1. Create a new view over events using standard SQL
  2. Create a new partitioned table using a standard SQL query
  3. Create a new view over events_partitioned using standard SQL
  4. Create a service account for the ODBC connection to use for authentication
  5. Create a Google Cloud Identity and Access Management (Cloud IAM) role for the ODBC connection and shared "events"
Show answer and explanation

Correct answer: C, D. Create a new view over events_partitioned using standard SQL; Create a service account for the ODBC connection to use for authentication

ODBC connections to BigQuery require standard SQL, not legacy SQL, so a new view or table must be created using standard SQL syntax. Option C creates a standard SQL view over the partitioned table, which replaces the legacy SQL view. Option D creates a service account for proper authentication from the ODBC connection, which is necessary for the application to connect securely. These two actions together ensure ODBC compatibility and secure authentication.

Why the other options are wrong

  • A. Creating a view over 'events' (which is itself a view) creates unnecessary indirection instead of directly referencing the source partitioned table.
  • B. Creating a partitioned table is unnecessary when a view suffices and adds maintenance overhead.
  • E. Cloud IAM roles are assigned to service accounts or users, not to connections themselves; the connection uses a service account for authentication.

Question 77

The marketing team at your organization provides regular updates of a segment of your customer dataset. The marketing team has given you a CSV with 1 million records that must be updated in BigQuery. When you use the UPDATE statement in BigQuery, you receive a quotaExceeded error. What should you do?

  1. Reduce the number of records updated each day to stay within the BigQuery UPDATE DML statement limit.
  2. Increase the BigQuery UPDATE DML statement limit in the Quota management section of the Google Cloud Platform Console.
  3. Split the source CSV file into smaller CSV files in Cloud Storage to reduce the number of BigQuery UPDATE DML statements per BigQuery job.
  4. Import the new records from the CSV file into a new BigQuery table. Create a BigQuery job that merges the new records with the existing records and writes the results to a new BigQuery table.
Show answer and explanation

Correct answer: D. Import the new records from the CSV file into a new BigQuery table. Create a BigQuery job that merges the new records with the existing records and writes the results to a new BigQuery table.

BigQuery UPDATE DML statements have hard limits on the number of rows they can modify in a single statement. Rather than trying to work around this limitation, the recommended approach is to use a merge pattern: import new data into a staging table, then use a merge operation to combine old and new records into a final table. This approach is more efficient, scalable, and follows BigQuery best practices for bulk updates.

Why the other options are wrong

  • A. Manually reducing records per day is a workaround that doesn't solve the problem and slows the update process.
  • B. DML quotas are not adjustable; they are hard limits enforced by BigQuery's architecture.
  • C. Splitting CSV files doesn't solve the underlying DML row limit issue, as each UPDATE statement still faces the same constraint.

Question 78

You store historic data in Cloud Storage. You need to perform analytics on the historic data. You want to use a solution to detect invalid data entries and perform data transformations that will not require programming or knowledge of SQL. What should you do?

  1. Use Cloud Dataflow with Beam to detect errors and perform transformations.
  2. Use Cloud Dataprep with recipes to detect errors and perform transformations.
  3. Use Cloud Dataproc with a Hadoop job to detect errors and perform transformations.
  4. Use federated tables in BigQuery with queries to detect errors and perform transformations.
Show answer and explanation

Correct answer: B. Use Cloud Dataprep with recipes to detect errors and perform transformations.

Cloud Dataprep is specifically designed for data quality and transformation tasks using a visual, recipe-based interface that requires no programming or SQL knowledge. It integrates seamlessly with Cloud Storage data sources and enables users to detect anomalies, clean data, and apply transformations through an intuitive UI.

Why the other options are wrong

  • A. Cloud Dataflow with Beam requires programming knowledge (Java or Python), contradicting the no-programming requirement.
  • C. Cloud Dataproc with Hadoop is a distributed processing framework requiring programming skills and Hadoop knowledge.
  • D. Federated tables in BigQuery still require SQL query writing for analysis and transformation, which contradicts the no-SQL requirement.

Question 79

You are building new real-time data warehouse for your company and will use Google BigQuery streaming inserts. There is no guarantee that data will only be sent in once but you do have a unique ID for each row of data and an event timestamp. You want to ensure that duplicates are not included while interactively querying data. Which query type should you use?

  1. Include ORDER BY DESK on timestamp column and LIMIT to 1.
  2. Use GROUP BY on the unique ID column and timestamp column and SUM on the values.
  3. Use the LAG window function with PARTITION by unique ID along with WHERE LAG IS NOT NULL.
  4. Use the ROW_NUMBER window function with PARTITION by unique ID along with WHERE row equals 1.
Show answer and explanation

Correct answer: D. Use the ROW_NUMBER window function with PARTITION by unique ID along with WHERE row equals 1.

The ROW_NUMBER window function with PARTITION BY unique ID assigns a sequential number to each row within the unique ID partition, ordered by timestamp. Using WHERE row_number = 1 keeps only the first occurrence (earliest timestamp) of each unique ID, effectively deduplicating the data. This works because the unique ID identifies logical duplicates and the timestamp orders them. Option A uses DESC which selects the latest timestamp incorrectly. Option B with GROUP BY and SUM aggregates values rather than deduplicating. Option C with LAG checks prior rows but doesn't properly eliminate duplicates when multiple copies arrive. ROW_NUMBER is the standard SQL deduplication pattern.

Why the other options are wrong

  • A. ORDER BY DESC with LIMIT 1 selects only the latest record globally, not deduplicating by unique ID.
  • B. GROUP BY with SUM aggregates values instead of deduplicating, changing data semantics.
  • C. LAG window function compares to previous rows but does not properly eliminate all duplicates across the dataset.

Question 80

Your company has a hybrid cloud initiative. You have a complex data pipeline that moves data between cloud provider services and leverages services from each of the cloud providers. Which cloud-native service should you use to orchestrate the entire pipeline?

  1. Cloud Dataflow
  2. Cloud Composer
  3. Cloud Dataprep
  4. Cloud Dataproc
Show answer and explanation

Correct answer: B. Cloud Composer

Cloud Composer is Google Cloud's managed Apache Airflow service designed specifically for orchestrating complex workflows across multiple cloud providers and on-premises systems. It supports hybrid cloud scenarios, allows defining DAGs (Directed Acyclic Graphs) for complex dependencies, and can trigger tasks across different cloud platforms and services.

Why the other options are wrong

  • A. Cloud Dataflow is for data processing pipelines, not for orchestrating workflows across multiple cloud providers.
  • C. Cloud Dataprep is for data cleaning and preparation, not for orchestrating complex multi-cloud pipelines.
  • D. Cloud Dataproc manages Hadoop/Spark clusters but is not designed for cross-cloud orchestration of entire pipelines.

Question 81

You've migrated a Hadoop job from an on-prem cluster to dataproc and GCS. Your Spark job is a complicated analytical workload that consists of many shuffling operations and initial data are parquet files (on average 200-400 MB size each). You see some degradation in performance after the migration to Dataproc, so you'd like to optimize for it. You need to keep in mind that your organization is very cost-sensitive, so you'd like to continue using Dataproc on preemptibles (with 2 non-preemptible workers only) for this workload. What should you do?

  1. Increase the size of your parquet files to ensure them to be 1 GB minimum.
  2. Switch to TFRecords formats (appr. 200MB per file) instead of parquet files.
  3. Switch from HDDs to SSDs, copy initial data from GCS to HDFS, run the Spark job and copy results back to GCS.
  4. Switch from HDDs to SSDs, override the preemptible VMs configuration to increase the boot disk size.
Show answer and explanation

Correct answer: D. Switch from HDDs to SSDs, override the preemptible VMs configuration to increase the boot disk size.

Shuffle-heavy Spark jobs write intermediate data to worker disks, and small HDD boot disks on preemptible workers give low IOPS and throughput. Switching to SSD persistent disks and overriding the preemptible configuration with larger boot disks speeds up shuffle while keeping the low-cost preemptible cluster.

Why the other options are wrong

  • A. The 200 to 400 MB Parquet files are already a reasonable size, and bigger files do not fix slow shuffle disks.
  • B. TFRecords are designed for TensorFlow, not Spark analytics, and would not improve shuffle performance.
  • C. HDFS on preemptible workers risks losing data when nodes are reclaimed, and copying data in and out adds time and complexity.

Question 82

Your analytics team wants to build a simple statistical model to determine which customers are most likely to work with your company again, based on a few different metrics. They want to run the model on Apache Spark, using data housed in Google Cloud Storage, and you have recommended using Google Cloud Dataproc to execute this job. Testing has shown that this workload can run in approximately 30 minutes on a 15-node cluster, outputting the results into Google BigQuery. The plan is to run this workload weekly. How should you optimize the cluster for cost?

  1. Migrate the workload to Google Cloud Dataflow
  2. Use preemptible virtual machines (VMs) for the cluster
  3. Use a higher-memory node so that the job runs faster
  4. Use SSDs on the worker nodes so that the job can run faster
Show answer and explanation

Correct answer: B. Use preemptible virtual machines (VMs) for the cluster

Preemptible VMs cost significantly less than standard VMs (typically 60-90% discount) and are ideal for batch workloads like this weekly 30-minute statistical model run. Since the job is fault-tolerant and doesn't require continuous uptime, preemptible VMs provide the best cost optimization. The workload is already optimized at 30 minutes on 15 nodes, so hardware upgrades won't improve cost-effectiveness.

Why the other options are wrong

  • A. Dataflow is designed for streaming and continuous data processing, not batch jobs, and wouldn't reduce costs for this use case.
  • C. Higher-memory nodes increase cost per node without addressing the fundamental cost optimization goal for a batch workload.
  • D. SSDs improve performance but add significant cost and don't align with the primary objective of cost optimization.

Question 83

You have several Spark jobs that run on a Cloud Dataproc cluster on a schedule. Some of the jobs run in sequence, and some of the jobs run concurrently. You need to automate this process. What should you do?

  1. Create a Cloud Dataproc Workflow Template
  2. Create an initialization action to execute the jobs
  3. Create a Directed Acyclic Graph in Cloud Composer
  4. Create a Bash script that uses the Cloud SDK to create a cluster, execute jobs, and then tear down the cluster
Show answer and explanation

Correct answer: C. Create a Directed Acyclic Graph in Cloud Composer

Cloud Composer is built on Apache Airflow and provides a managed workflow orchestration service designed for scheduling and managing complex job dependencies. It natively supports directed acyclic graphs (DAGs) that represent sequential and concurrent job execution patterns, making it ideal for automating Spark jobs that run both in sequence and concurrently. Composer integrates seamlessly with Cloud Dataproc clusters and handles scheduling, monitoring, and retry logic.

Why the other options are wrong

  • A. Cloud Dataproc Workflow Templates execute jobs on a single cluster but don't provide the orchestration capabilities needed to manage complex dependencies and concurrent execution patterns across multiple jobs.
  • B. Initialization actions run at cluster startup and are designed for environment setup, not for orchestrating scheduled job execution with dependencies.
  • D. While a custom Bash script could technically work, it lacks production-grade features like monitoring, retry logic, dependency management, and is difficult to maintain and scale.

Question 84

Your company's customer and order databases are often under heavy load. This makes performing analytics against them difficult without harming operations. The databases are in a MySQL cluster, with nightly backups taken using mysqldump. You want to perform analytics with minimal impact on operations. What should you do?

  1. Add a node to the MySQL cluster and build an OLAP cube there.
  2. Use an ETL tool to load the data from MySQL into Google BigQuery.
  3. Connect an on-premises Apache Hadoop cluster to MySQL and perform ETL.
  4. Mount the backups to Google Cloud SQL, and then process the data using Google Cloud Dataproc.
Show answer and explanation

Correct answer: B. Use an ETL tool to load the data from MySQL into Google BigQuery.

Using an ETL tool to load data from MySQL into Google BigQuery provides the best solution for analytics with minimal operational impact. BigQuery is a fully managed data warehouse that handles analytics workloads without burdening the production MySQL cluster. The nightly backups provide a natural point for ETL, and BigQuery scales automatically for complex queries without affecting the operational database performance.

Why the other options are wrong

  • A. Adding a node to the MySQL cluster for OLAP cubes still keeps the data within the same operational infrastructure, risking performance degradation and requiring ongoing maintenance.
  • C. An on-premises Hadoop cluster requires maintaining separate infrastructure and moving large data volumes over the network, defeating the purpose of reducing operational burden.
  • D. Mounting backups to Cloud SQL and using Dataproc is unnecessarily complex; Cloud SQL is still a relational database not optimized for analytics, and Dataproc adds operational overhead compared to BigQuery.

Question 85

You are managing a Cloud Dataproc cluster. You need to make a job run faster while minimizing costs, without losing work in progress on your clusters. What should you do?

  1. Increase the cluster size with more non-preemptible workers.
  2. Increase the cluster size with preemptible worker nodes, and configure them to forcefully decommission.
  3. Increase the cluster size with preemptible worker nodes, and use Cloud Stackdriver to trigger a script to preserve work.
  4. Increase the cluster size with preemptible worker nodes, and configure them to use graceful decommissioning.
Show answer and explanation

Correct answer: D. Increase the cluster size with preemptible worker nodes, and configure them to use graceful decommissioning.

Preemptible worker nodes cost 60-90% less than standard nodes, directly minimizing cost. Graceful decommissioning is critical for protecting work in progress, when a preemptible node is reclaimed, it enters graceful decommissioning mode, giving running tasks time to complete or checkpoint before the node is forcibly removed. This ensures no work is lost while still gaining the performance benefit of additional compute capacity at a fraction of the cost. Non-preemptible nodes do not achieve the cost minimization goal, and forceful decommissioning would cause immediate work loss.

Why the other options are wrong

  • A. Non-preemptible workers do not minimize costs; they are significantly more expensive than preemptible alternatives.
  • B. Forced decommissioning causes immediate termination of running tasks, resulting in loss of work in progress.
  • C. Using Stackdriver to trigger manual preservation scripts is operationally complex and unreliable compared to native graceful decommissioning.

Question 86

You are creating a model to predict housing prices. Due to budget constraints, you must run it on a single resource-constrained virtual machine. Which learning algorithm should you use?

  1. Linear regression
  2. Logistic classification
  3. Recurrent neural network
  4. Feedforward neural network
Show answer and explanation

Correct answer: A. Linear regression

Linear regression is a lightweight algorithm requiring minimal computational resources and memory, making it ideal for resource-constrained environments. It trains quickly and scales efficiently even on single VMs. Logistic classification is also simple but is designed for classification, not price prediction (a regression task). Recurrent neural networks require significant memory and compute for sequence processing. Feedforward neural networks, while general-purpose, require more resources than linear models for effective training. For budget-constrained scenarios with regression problems, linear regression provides the best accuracy-to-resource ratio.

Why the other options are wrong

  • B. Logistic classification is for classification tasks, not housing price prediction which is a regression problem.
  • C. Recurrent neural networks require substantial memory and processing power unsuitable for resource-constrained VMs.
  • D. Feedforward neural networks require more computational resources than necessary for a single resource-constrained VM.

Question 87

You use a dataset in BigQuery for analysis. You want to provide third-party companies with access to the same dataset. You need to keep the costs of data sharing low and ensure that the data is current. Which solution should you choose?

  1. Use Analytics Hub to control data access, and provide third party companies with access to the dataset.
  2. Use Cloud Scheduler to export the data on a regular basis to Cloud Storage, and provide third-party companies with access to the bucket.
  3. Create a separate dataset in BigQuery that contains the relevant data to share, and provide third-party companies with access to the new dataset.
  4. Create a Dataflow job that reads the data in frequent time intervals, and writes it to the relevant BigQuery dataset or Cloud Storage bucket for third-party companies to use.
Show answer and explanation

Correct answer: A. Use Analytics Hub to control data access, and provide third party companies with access to the dataset.

Analytics Hub is Google Cloud's solution for secure, low-cost data sharing. It allows direct access to datasets without copying data, which keeps costs minimal and ensures data is always current since third parties query the live dataset. It provides fine-grained access controls and eliminates the need for data replication or export processes.

Why the other options are wrong

  • B. Exporting to Cloud Storage and requiring manual access management adds latency, introduces staleness, and increases operational complexity.
  • C. Copying data to a separate dataset duplicates storage costs and creates maintenance overhead for keeping data synchronized.
  • D. Using Dataflow for frequent replication adds compute costs and introduces latency, defeating the goal of keeping costs low and data current.

That was 87 of 349.

The full Google Professional Data Engineer pack has all 349 questions, each with the answer, the explanation and why the other options are wrong, plus a questions-only copy for timed runs. US$39, paid once, with free monthly updates and a pass-or-your-money-back guarantee.

Get the full pack

Not ready yet? Get the first 138 free as one PDF by email. Get the Half Pack →