Google Exam Professional-Machine-Learning-Engineer Topic 4 Question 74 Discussion

Actual exam question for Google's Google Professional Machine Learning Engineer exam

Question #: 74
Topic #: 4

[All Google Professional Machine Learning Engineer Questions]

You have trained an XGBoost model that you plan to deploy on Vertex Al for online prediction. You are now uploading your model to Vertex Al Model Registry, and you need to configure the explanation method that will serve online prediction requests to be returned with minimal latency. You also want to be alerted when feature attributions of the model meaningfully change over time. What should you do?

A1 Specify sampled Shapley as the explanation method with a path count of 5.
2 Deploy the model to Vertex Al Endpoints.
3. Create a Model Monitoring job that uses prediction drift as the monitoring objective.

B1 Specify Integrated Gradients as the explanation method with a path count of 5.
2 Deploy the model to Vertex Al Endpoints.
3. Create a Model Monitoring job that uses prediction drift as the monitoring objective.

C1. Specify sampled Shapley as the explanation method with a path count of 50.
2. Deploy the model to Vertex Al Endpoints.
3. Create a Model Monitoring job that uses training-serving skew as the monitoring objective.

D1 Specify Integrated Gradients as the explanation method with a path count of 50.
2. Deploy the model to Vertex Al Endpoints.
3 Create a Model Monitoring job that uses training-serving skew as the monitoring objective.

Show Suggested Answer

Suggested Answer: A

Sampled Shapley is a fast and scalable approximation of the Shapley value, which is a game-theoretic concept that measures the contribution of each feature to the model prediction. Sampled Shapley is suitable for online prediction requests, as it can return feature attributions with minimal latency. The path count parameter controls the number of samples used to estimate the Shapley value, and a lower value means faster computation. Integrated Gradients is another explanation method that computes the average gradient along the path from a baseline input to the actual input. Integrated Gradients is more accurate than Sampled Shapley, but also more computationally intensive. Therefore, it is not recommended for online prediction requests, especially with a high path count. Prediction drift is the change in the distribution of feature values or labels over time. It can affect the performance and accuracy of the model, and may require retraining or redeploying the model. Vertex AI Model Monitoring allows you to monitor prediction drift on your deployed models and endpoints, and set up alerts and notifications when the drift exceeds a certain threshold. You can specify an email address to receive the notifications, and use the information to retrigger the training pipeline and deploy an updated version of your model. This is the most direct and convenient way to achieve your goal. Training-serving skew is the difference between the data used for training the model and the data used for serving the model. It can also affect the performance and accuracy of the model, and may indicate data quality issues or model staleness. Vertex AI Model Monitoring allows you to monitor training-serving skew on your deployed models and endpoints, and set up alerts and notifications when the skew exceeds a certain threshold. However, this is not relevant to the question, as the question is about the feature attributions of the model, not the data distribution.Reference:

Vertex AI: Explanation methods

Vertex AI: Configuring explanations

Vertex AI: Monitoring prediction drift

Vertex AI: Monitoring training-serving skew

by Sharen at Apr 10, 2024, 03:07 AM

Limited Time Offer

25%

Off

Get Premium Professional-Machine-Learning-Engineer Questions as Interactive Web-Based Practice Test or PDF

Comments

Submit Cancel

Pura

1 days ago

Hmm, let's think this through. I think the key is to choose an explanation method that can provide meaningful feature attributions with minimal latency for the online prediction requests. And we also want to set up monitoring to detect changes in the feature attributions over time.

upvoted 0 times

...

Polly

3 days ago

Wait, so we need to pick the right explanation method and monitoring objective for this XGBoost model deployment on Vertex AI? This seems like a tricky question. I'm not sure if I fully understand the different explanation methods and monitoring objectives mentioned.

upvoted 0 times

...