According to the official exam guide1, one of the skills assessed in the exam is to “automate and

orchestrate ML pipelines using Cloud Composer”. Cloud Composer2 is a fully managed workflow

orchestration service that uses Apache Airflow to create, schedule, monitor, and manage workflows.

Cloud Composer allows you to build custom training pipelines for deep learning-based models and

integrate them with other Google Cloud services. You can also use Cloud Composer to implement

model interpretability techniques, such as feature attributions, explainable AI, or model debugging3.

The other options are not relevant or optimal for this scenario. Reference:

Professional ML Engineer Exam Guide

Cloud Composer

Model interpretability with Cloud Composer

Google Professional Machine Learning Certification Exam 2023

Latest Google Professional Machine Learning Engineer Actual Free Exam Questions

According to the official exam guide1, one of the skills assessed in the exam is to “track the lineage

of pipeline artifacts”. Vertex ML Metadata2 is a service that allows you to store, query, and visualize

metadata associated with your ML workflows, such as datasets, models, metrics, and executions.

Vertex ML Metadata helps you track the provenance and lineage of your ML artifacts and understand

the relationships between them. Vertex AI Experiments3 is a service that allows you to track and

compare the results of your model training runs. Vertex AI Experiments automatically logs metadata

such as hyperparameters, metrics, and artifacts for each training run. You can use Vertex AI

Experiments to train your custom model using TensorFlow, PyTorch, XGBoost, or scikit-learn. Vertex

AI TensorBoard4 is a service that allows you to visualize and monitor your ML experiments using

TensorBoard, an open source tool for ML visualization. Vertex AI TensorBoard helps you track the

model’s training parameters and the metrics per training epoch, and compare the performance of

each version of the model. Therefore, option C is the best way to determine which Vertex AI services

to include in the workflow for the given use case. The other options are not relevant or optimal for

this scenario. Reference:

Professional ML Engineer Exam Guide

Vertex ML Metadata

Vertex AI Experiments

Vertex AI TensorBoard

Google Professional Machine Learning Certification Exam 2023

Latest Google Professional Machine Learning Engineer Actual Free Exam Questions

The best option for using Vertex AI Model Monitoring for drift detection and minimizing the cost is to

use the features and the feature attributions for monitoring, and set a prediction-sampling-rate value

that is closer to 0 than 1. This option allows you to leverage the power and flexibility of Google Cloud

to detect feature drift in the input predict requests for custom models, and reduce the storage and

computation costs of the model monitoring job. Vertex AI Model Monitoring is a service that can

track and compare the results of multiple machine learning runs. Vertex AI Model Monitoring can

monitor the model’s prediction input data for feature skew and drift. Feature drift occurs when the

feature data distribution in production changes over time. If the original training data is not available,

you can enable drift detection to monitor your models for feature drift. Vertex AI Model Monitoring

uses TensorFlow Data Validation (TFDV) to calculate the distributions and distance scores for each

feature, and compares them with a baseline distribution. The baseline distribution is the statistical

distribution of the feature’s values in the training data. If the training data is not available, the

baseline distribution is calculated from the first 1000 prediction requests that the model receives. If

the distance score for a feature exceeds an alerting threshold that you set, Vertex AI Model

Monitoring sends you an email alert. However, if you use a custom model, you can also enable

feature attribution monitoring, which can provide more insights into the feature drift. Feature

attribution monitoring analyzes the feature attributions, which are the contributions of each feature

to the prediction output. Feature attribution monitoring can help you identify the features that have

the most impact on the model performance, and the features that have the most significant drift

over time. Feature attribution monitoring can also help you understand the relationship between the

features and the prediction output, and the correlation between the features1. The prediction-

sampling-rate is a parameter that determines the percentage of prediction requests that are logged

and analyzed by the model monitoring job. Using a lower prediction-sampling-rate can reduce the

storage and computation costs of the model monitoring job, but also the quality and validity of the

data. Using a lower prediction-sampling-rate can introduce sampling bias and noise into the data,

and make the model monitoring job miss some important features or patterns of the data. However,

using a higher prediction-sampling-rate can increase the storage and computation costs of the model

monitoring job, and also the amount of data that needs to be processed and analyzed. Therefore,

there is a trade-off between the prediction-sampling-rate and the cost and accuracy of the model

monitoring job, and the optimal prediction-sampling-rate depends on the business objective and the

data characteristics2. By using the features and the feature attributions for monitoring, and setting a

prediction-sampling-rate value that is closer to 0 than 1, you can use Vertex AI Model Monitoring for

drift detection and minimize the cost.

The other options are not as good as option D, for the following reasons:

Option A: Using the features for monitoring and setting a monitoring-frequency value that is higher

than the default would not enable feature attribution monitoring, and could increase the cost of the

model monitoring job. The monitoring-frequency is a parameter that determines how often the

model monitoring job analyzes the logged prediction requests and calculates the distributions and

distance scores for each feature. Using a higher monitoring-frequency can increase the frequency

and timeliness of the model monitoring job, but also the computation costs of the model monitoring

job. Moreover, using the features for monitoring would not enable feature attribution monitoring,

which can provide more insights into the feature drift and the model performance1.

Option B: Using the features for monitoring and setting a prediction-sampling-rate value that is

closer to 1 than 0 would not enable feature attribution monitoring, and could increase the cost of the

model monitoring job. The prediction-sampling-rate is a parameter that determines the percentage

of prediction requests that are logged and analyzed by the model monitoring job. Using a higher

prediction-sampling-rate can increase the quality and validity of the data, but also the storage and

computation costs of the model monitoring job. Moreover, using the features for monitoring would

not enable feature attribution monitoring, which can provide more insights into the feature drift and

the model performance12.

Option C: Using the features and the feature attributions for monitoring and setting a monitoring-

frequency value that is lower than the default would enable feature attribution monitoring, but

could reduce the frequency and timeliness of the model monitoring job. The monitoring-frequency is

a parameter that determines how often the model monitoring job analyzes the logged prediction

requests and calculates the distributions and distance scores for each feature. Using a lower

monitoring-frequency can reduce the computation costs of the model monitoring job, but also the

frequency and timeliness of the model monitoring job. This can make the model monitoring job less

responsive and effective in detecting and alerting the feature drift1.

Reference:

Preparing for Google Cloud Certification: Machine Learning Engineer, Course 3: Production ML

Systems, Week 4: Evaluation

Google Cloud Professional Machine Learning Engineer Exam Guide, Section 3: Scaling ML models in

production, 3.3 Monitoring ML models in production

Official Google Cloud Certified Professional Machine Learning Engineer Study Guide, Chapter 6:

Production ML Systems, Section 6.3: Monitoring ML Models

Using Model Monitoring

Understanding the score threshold slider

Cloud Vision API is a service that allows you to analyze images using pre-trained machine learning

models1. You can use Cloud Vision API to perform various tasks, such as face detection, text

extraction, logo recognition, and object localization1. Object localization is a feature that allows you

to detect multiple objects in an image and draw bounding boxes around them2. You can also get the

labels and confidence scores for each detected object2.

By sending user-submitted images to the Cloud Vision API, you can use object localization to identify

all objects in the image and compare the results against a list of animals. You can use

the OBJECT_LOCALIZATION feature type in the AnnotateImageRequest to request object

localization3. You can then use the localizedObjectAnnotations field in

the AnnotateImageResponse to get the list of detected objects, their labels, and their confidence

scores. You can compare the labels with a predefined list of animals, such as dogs, cats, birds, etc.,

and determine whether the image has an animal or not.

This option is the best for your scenario, because it allows you to automatically and in near real time

detect whether each uploaded photo has an animal, without requiring any manual labeling, model

training, or model deployment. You can also prioritize time and minimize cost of your application

development and deployment, as you can use the Cloud Vision API as a ready-to-use service, without

needing any machine learning expertise or infrastructure.

The other options are not suitable for your scenario, because they either require manual labeling,

model training, or model deployment, which would increase the time and cost of your application

development and deployment, or they use object detection models, which are more complex and

computationally expensive than object localization models, and are not necessary for your simple

task of detecting whether an image has an animal or not.

Reference:

Cloud Vision API | Google Cloud

Object localization | Cloud Vision API | Google Cloud

AnnotateImageRequest | Cloud Vision API | Google Cloud

[AnnotateImageResponse | Cloud Vision API | Google Cloud]

Option A is incorrect because distributing the dataset with

tf.distribute.Strategy.experimental_distribute_dataset is not the most effective way to decrease the

training time. This method allows you to distribute your dataset across multiple devices or machines,

by creating a tf.data.Dataset instance that can be iterated over in parallel1. However, this option may

not improve the training time significantly, as it does not change the amount of data or computation

that each device or machine has to process. Moreover, this option may introduce additional

overhead or complexity, as it requires you to handle the data sharding, replication, and

synchronization across the devices or machines1.

Option B is incorrect because creating a custom training loop is not the easiest way to decrease the

training time. A custom training loop is a way to implement your own logic for training your model,

by using low-level TensorFlow APIs, such as tf.GradientTape, tf.Variable, or tf.function2. A custom

training loop may give you more flexibility and control over the training process, but it also requires

more effort and expertise, as you have to write and debug the code for each step of the training loop,

such as computing the gradients, applying the optimizer, or updating the metrics2. Moreover, a

custom training loop may not improve the training time significantly, as it does not change the

amount of data or computation that each device or machine has to process.

Option C is incorrect because using a TPU with tf.distribute.TPUStrategy is not a valid way to decrease

the training time. A TPU (Tensor Processing Unit) is a custom hardware accelerator designed for high-

performance ML workloads3. A tf.distribute.TPUStrategy is a distribution strategy that allows you to

distribute your training across multiple TPUs, by creating a tf.distribute.TPUStrategy instance that can

be used with high-level TensorFlow APIs, such as Keras4. However, this option is not feasible, as

Vertex AI Training does not support TPUs as accelerators for custom training jobs5. Moreover, this

option may require significant code changes, as TPUs have different requirements and limitations

than GPUs.

Option D is correct because increasing the batch size is the best way to decrease the training time.

The batch size is a hyperparameter that determines how many samples of data are processed in each

iteration of the training loop. Increasing the batch size may reduce the training time, as it reduces the

number of iterations needed to train the model, and it allows each device or machine to process

more data in parallel. Increasing the batch size is also easy to implement, as it only requires changing

a single hyperparameter. However, increasing the batch size may also affect the convergence and the

accuracy of the model, so it is important to find the optimal batch size that balances the trade-off

between the training time and the model performance.

Reference:

tf.distribute.Strategy.experimental_distribute_dataset

Custom training loop

TPU overview

tf.distribute.TPUStrategy

Vertex AI Training accelerators

[TPU programming model]

[Batch size and learning rate]

[Keras overview]

[tf.distribute.MirroredStrategy]

[Vertex AI Training overview]

[TensorFlow overview]

A ResourceExhaustedError: out of memory (OOM) when allocating tensor is an error that occurs

when the GPU runs out of memory while trying to allocate memory for a tensor. A tensor is a multi-

dimensional array of numbers that represents the data or the parameters of a machine learning

model. The size and shape of a tensor depend on various factors, such as the input data, the model

architecture, the batch size, and the optimization algorithm1.

For the use case of training a computer vision model that predicts the type of government ID present

in a given image using a GPU-powered virtual machine on Compute Engine, the best option to

resolve the error is to reduce the batch size. The batch size is a parameter that determines how many

input examples are processed at a time by the model. A larger batch size can improve the model’s

accuracy and stability, but it also requires more memory and computation. A smaller batch size can

reduce the memory and computation requirements, but it may also affect the model’s performance

and convergence2.

By reducing the batch size, the GPU can allocate less memory for each tensor, and avoid running out

of memory. Reducing the batch size can also speed up the training process, as the GPU can process

more batches in parallel. However, reducing the batch size too much may also have some drawbacks,

such as increasing the noise and variance of the gradient updates, and slowing down the

convergence of the model. Therefore, the optimal batch size should be chosen based on the trade-off

between memory, computation, and performance3.

The other options are not as effective as option B, because they are not directly related to the

memory allocation of the GPU. Option A, changing the optimizer, may affect the speed and quality of

the optimization process, but it may not reduce the memory usage of the model. Option C, changing

the learning rate, may affect the convergence and stability of the model, but it may not reduce the

memory usage of the model. Option D, reducing the image shape, may reduce the size of the input

tensor, but it may also reduce the quality and resolution of the image, and affect the model’s

accuracy. Therefore, option B, reducing the batch size, is the best answer for this question.

Reference:

ResourceExhaustedError: OOM when allocating tensor with shape - Stack Overflow

How does batch size affect model performance and training time? - Stack Overflow

How to choose an optimal batch size for training a neural network? - Stack Overflow

Option A is incorrect because Vertex AI Pipelines and App Engine do not meet all the requirements of

the system. Vertex AI Pipelines is a service that allows you to create, run, and manage ML workflows

using TensorFlow Extended (TFX) components or custom components1. App Engine is a service that

allows you to build and deploy scalable web applications using standard or flexible

environments2. However, App Engine does not support Docker containers in the standard

environment, and does not provide a dedicated service for online prediction and monitoring of ML

models3.

Option B is correct because Vertex AI Pipelines, Vertex AI Prediction, and Vertex AI Model Monitoring

meet all the requirements of the system. Vertex AI Prediction is a service that allows you to deploy

and serve ML models for online or batch prediction, with support for autoscaling and custom

containers4. Vertex AI Model Monitoring is a service that allows you to monitor the performance and

fairness of your deployed models, and get alerts for any issues or anomalies5.

Option C is incorrect because Cloud Composer, BigQuery ML, and Vertex AI Prediction do not meet

all the requirements of the system. Cloud Composer is a service that allows you to create, schedule,

and manage workflows using Apache Airflow. BigQuery ML is a service that allows you to create and

use ML models within BigQuery using SQL queries. However, BigQuery ML does not support custom

containers, and Vertex AI Prediction does not support scheduled model retraining or model

monitoring.

Option D is incorrect because Cloud Composer, Vertex AI Training with custom containers, and App

Engine do not meet all the requirements of the system. Vertex AI Training is a service that allows you

to train ML models using built-in algorithms or custom containers. However, Vertex AI Training does

not support online prediction or model monitoring, and App Engine does not support Docker

containers in the standard environment or online prediction and monitoring of ML models3.

Reference:

Vertex AI Pipelines overview

App Engine overview

Choosing an App Engine environment

Vertex AI Prediction overview

Vertex AI Model Monitoring overview

[Cloud Composer overview]

[BigQuery ML overview]

[BigQuery ML limitations]

[Vertex AI Training overview]

According to the official exam guide1, one of the skills assessed in the exam is to “configure and

optimize model monitoring jobs”. Vertex AI Model Monitoring documentation states that “model

monitoring helps you detect when your model’s performance degrades over time due to changes in

the data that your model receives or returns” and that "you can configure model monitoring to send

notifications to Pub/Sub when it detects anomalies or drift in your model’s predictions"2. Therefore,

enabling model monitoring on the Vertex AI endpoint and configuring Pub/Sub to call the Cloud

Function when feature drift is detected would help you keep the model up-to-date and minimize

retraining costs. The other options are not relevant or optimal for this scenario. Reference:

Professional ML Engineer Exam Guide

Vertex AI Model Monitoring

Google Professional Machine Learning Certification Exam 2023

Latest Google Professional Machine Learning Engineer Actual Free Exam Questions

Kubeflow is an open source platform for developing, orchestrating, deploying, and running scalable

and portable machine learning workflows on Kubernetes. Kubeflow Pipelines is a component of

Kubeflow that allows you to build and manage end-to-end machine learning pipelines using a

graphical user interface or a Python-based domain-specific language (DSL). Kubeflow Pipelines can

help you automate and orchestrate your machine learning workflows, and integrate with various

Google Cloud services and tools1

One of the Google Cloud services that you can use with Kubeflow Pipelines is BigQuery, which is a

serverless, scalable, and cost-effective data warehouse that allows you to run fast and complex

queries on large-scale data. BigQuery can help you analyze and prepare your data for machine

learning, and store and manage your machine learning models2

To execute a query against BigQuery as the first step in your Kubeflow pipeline, and use the results of

that query as the input to the next step in your pipeline, the easiest way to do that is to use the

BigQuery Query Component, which is a pre-built component that you can find in the Kubeflow

Pipelines repository on GitHub. The BigQuery Query Component allows you to run a SQL query on

BigQuery, and output the results as a table or a file. You can use the component’s URL to load the

component into your pipeline, and specify the query and the output parameters. You can then use

the output of the component as the input to the next step in your pipeline, such as a data processing

or a model training step3

The other options are not as easy or feasible. Using the BigQuery console to execute your query and

then save the query results into a new BigQuery table is not a good idea, as it does not integrate with

your Kubeflow pipeline, and requires manual intervention and duplication of data. Writing a Python

script that uses the BigQuery API to execute queries against BigQuery is not ideal, as it requires

writing custom code and handling authentication and error handling. Using the Kubeflow Pipelines

DSL to create a custom component that uses the Python BigQuery client library to execute queries is

not optimal, as it requires creating and packaging a Docker container image for the component, and

testing and debugging the component.

Reference: 1: Kubeflow Pipelines overview 2: BigQuery overview 3: BigQuery Query Component

The most likely problem is that the tables that you created to hold your training and validation

records share some records, and you may not be using all the data in your initial table. This is

because the RAND() function generates a random number between 0 and 1 for each row, and the

probability of a row being in both the training and validation tables is 0.2 * 0.8 0.16, which is not

negligible. This means that some of the records that you use to validate your model are also used to

train your model, which can lead to overfitting and poor generalization. Moreover, the probability of

a row being in neither the training nor the validation table is 0.2 * 0.2 0.04, which means that you

are wasting some of the data in your initial table and reducing the size of your datasets. A better way

to split your data into training and validation sets is to use a hash function on a unique identifier

column, such as the following queries:

CREATE OR REPLACE TABLE ‘myproject.mydataset.training‘ AS (SELECT * FROM

‘myproject.mydataset.mytable‘ WHERE MOD(FARM_FINGERPRINT(id), 10) < 8); CREATE OR REPLACE

TABLE ‘myproject.mydataset.validation‘ AS (SELECT * FROM ‘myproject.mydataset.mytable‘ WHERE

MOD(FARM_FINGERPRINT(id), 10) > 8);

This way, you can ensure that each row has a fixed 80% chance of being in the training table and a

20% chance of being in the validation table, without any overlap or omission.

Reference:

Professional ML Engineer Exam Guide

Preparing for Google Cloud Certification: Machine Learning Engineer Professional Certificate

Google Cloud launches machine learning engineer certification

BigQuery ML: Splitting data for training and testing

BigQuery: FARM_FINGERPRINT function