Deal of The Day! Hurry Up, Grab the Special Discount - Save 25% - Ends In 00:00:00 Coupon code: SAVE25
Welcome to Pass4Success

- Free Preparation Discussions
Mail Us support@pass4success.com
Location US
Mob_nav_barsMENU

Google Professional Machine Learning Engineer Exam - Topic 3 Question 91 Discussion

You work for an organization that operates a streaming music service. You have a custom production model that is serving a "next song" recommendation based on a user's recent listening history. Your model is deployed on a Vertex Al endpoint. You recently retrained the same model by using fresh data. The model received positive test results offline. You now want to test the new model in production while minimizing complexity. What should you do?
C) Deploy the new model to the existing Vertex Al endpoint Use traffic splitting to send 5% of production traffic to the new model Monitor end-user metrics, such as listening time If end-user metrics improve between models over time, gradually increase the percentage of production traffic sent to the new model.
A) Create a new Vertex Al endpoint for the new model and deploy the new model to that new endpoint Build a service to randomly send 5% of production traffic to the new endpoint Monitor end-user metrics such as listening time If end-user metrics improve between models over time gradually increase the percentage of production traffic sent to the new endpoint.
B) Capture incoming prediction requests in BigQuery Create an experiment in Vertex Al Experiments Run batch predictions for both models using the captured data Use the user's selected song to compare the models performance side by side If the new models performance metrics are better than the previous model deploy the new model to production.
D) Configure a model monitoring job for the existing Vertex Al endpoint. Configure the monitoring job to detect prediction drift, and set a threshold for alerts Update the model on the endpoint from the previous model to the new model If you receive an alert of prediction drift, revert to the previous model.

Google Professional Machine Learning Engineer Exam - Topic 3 Question 91 Discussion

Actual exam question for Google's Professional Machine Learning Engineer exam
Question #: 91
Topic #: 3
[All Professional Machine Learning Engineer Questions]

You work for an organization that operates a streaming music service. You have a custom production model that is serving a "next song" recommendation based on a user's recent listening history. Your model is deployed on a Vertex Al endpoint. You recently retrained the same model by using fresh dat

a. The model received positive test results offline. You now want to test the new model in production while minimizing complexity. What should you do?

Show Suggested Answer Hide Answer
Suggested Answer: C

Traffic splitting is a feature of Vertex AI that allows you to distribute the prediction requests among multiple models or model versions within the same endpoint. You can specify the percentage of traffic that each model or model version receives, and change it at any time. Traffic splitting can help you test the new model in production without creating a new endpoint or a separate service. You can deploy the new model to the existing Vertex AI endpoint, and use traffic splitting to send 5% of production traffic to the new model. You can monitor the end-user metrics, such as listening time, to compare the performance of the new model and the previous model. If the end-user metrics improve between models over time, you can gradually increase the percentage of production traffic sent to the new model. This solution can help you test the new model in production while minimizing complexity and cost.Reference:

Traffic splitting | Vertex AI

Deploying models to endpoints | Vertex AI


by Selma at Sep 16, 2024, 08:52 PM

Limited Time Offer

25%

Off

Get Premium Professional Machine Learning Engineer Questions as Interactive Web-Based Practice Test or PDF

Contribute your Thoughts:

0/2000 characters
Submit Cancel
Evelynn
9 months ago
I agree with C, traffic splitting is smart!
upvoted 0 times
...
Leonardo
9 months ago
Not sure if monitoring alone is enough, D feels risky.
upvoted 0 times
...
Allene
10 months ago
Surprised there's no mention of A/B testing directly!
upvoted 0 times
...
Hui
10 months ago
I think A is better for minimizing risk.
upvoted 0 times
...
Shenika
10 months ago
Option C seems like a solid choice for gradual testing.
upvoted 0 times
...
Floyd
10 months ago
Option D seems like a safe choice with the monitoring job, but I wonder if it really minimizes complexity as much as the others.
upvoted 0 times
...
Suzan
11 months ago
I feel like option B might be more thorough since it involves comparing models side by side, but it seems a bit more complex than what we practiced.
upvoted 0 times
...
Quentin
11 months ago
I think option A is similar to a practice question we did about gradual rollout, but I'm not confident about the specifics of monitoring end-user metrics.
upvoted 0 times
...
Kristal
11 months ago
I remember we discussed traffic splitting in class, so option C sounds familiar, but I'm not entirely sure if it's the best approach for minimizing complexity.
upvoted 0 times
...
Noel
11 months ago
I'm feeling pretty confident about this one. Option A seems like the cleanest approach to gradually test the new model in production. The key is starting small with 5% of traffic and closely monitoring the metrics before scaling up. That way you can minimize the risk.
upvoted 0 times
...
Julie
11 months ago
I'm a bit confused by the options. Option B seems like a lot of extra work with the BigQuery and Vertex AI Experiments. I think the simpler approach in option C would be better - just use traffic splitting on the existing endpoint.
upvoted 0 times
...
Annmarie
11 months ago
This seems like a straightforward A/B testing scenario. I'd go with option A - create a new endpoint, send a small percentage of traffic to the new model, and gradually increase it if the metrics improve.
upvoted 0 times
...
Maryann
11 months ago
Hmm, option D with the model monitoring and drift detection sounds interesting. That could be a good way to automatically detect issues with the new model and roll back if needed. I might consider that if I'm worried about the impact of a full model swap.
upvoted 0 times
...
Jeanice
11 months ago
Based on past practice questions, FBX definitely stands out for preserving animation data, so I'm leaning that way.
upvoted 0 times
...
Nakita
2 years ago
I prefer option B. Capturing incoming prediction requests in BigQuery and running batch predictions for both models seems like a thorough approach to compare performance.
upvoted 0 times
...
Thomasena
2 years ago
I'm partial to B - I love a good data-driven experiment! Although, I hope they're not using the 'song selected' metric as the only KPI. Gotta look at the whole user experience.
upvoted 0 times
...
Tasia
2 years ago
Option C is the way to go - it's the 'Goldilocks' solution, not too disruptive, not too risky. Just right!
upvoted 0 times
...
Daren
2 years ago
A reminds me of A/B testing, which is a classic approach. I wonder if the random 5% split might lead to some users getting a suboptimal experience though.
upvoted 0 times
Tashia
2 years ago
A) I agree, A/B testing is a common method. But you're right, there could be some users who might not have the best experience with the 5% split.
upvoted 0 times
...
Isadora
2 years ago
C) Deploy the new model to the existing Vertex Al endpoint Use traffic splitting to send 5% of production traffic to the new model Monitor end-user metrics, such as listening time If end-user metrics improve between models over time, gradually increase the percentage of production traffic sent to the new model.
upvoted 0 times
...
Josue
2 years ago
B) Capture incoming prediction requests in BigQuery Create an experiment in Vertex Al Experiments Run batch predictions for both models using the captured data Use the user's selected song to compare the models performance side by side If the new models performance metrics are better than the previous model deploy the new model to production.
upvoted 0 times
...
Melynda
2 years ago
A) Create a new Vertex Al endpoint for the new model and deploy the new model to that new endpoint Build a service to randomly send 5% of production traffic to the new endpoint Monitor end-user metrics such as listening time If end-user metrics improve between models over time gradually increase the percentage of production traffic sent to the new endpoint.
upvoted 0 times
...
...
Vincenza
2 years ago
D is a clever idea, using monitoring to automatically update the model. But I'm not sure I'd trust the drift detection to work perfectly on the first try.
upvoted 0 times
Vince
2 years ago
D) Configure a model monitoring job for the existing Vertex AI endpoint. Configure the monitoring job to detect prediction drift, and set a threshold for alerts. Update the model on the endpoint from the previous model to the new model. If you receive an alert of prediction drift, revert to the previous model.
upvoted 0 times
...
Candra
2 years ago
C) Deploy the new model to the existing Vertex AI endpoint. Use traffic splitting to send 5% of production traffic to the new model. Monitor end-user metrics, such as listening time. If end-user metrics improve between models over time, gradually increase the percentage of production traffic sent to the new model.
upvoted 0 times
...
Arlean
2 years ago
A) Create a new Vertex AI endpoint for the new model and deploy the new model to that new endpoint. Build a service to randomly send 5% of production traffic to the new endpoint. Monitor end-user metrics such as listening time. If end-user metrics improve between models over time, gradually increase the percentage of production traffic sent to the new endpoint.
upvoted 0 times
...
...
Miesha
2 years ago
I agree with Lorrie. Option A seems like the most efficient way to test the new model while minimizing complexity.
upvoted 0 times
...
Lorrie
2 years ago
I think option A is the best choice. It allows for gradual testing of the new model in production.
upvoted 0 times
...
Ling
2 years ago
B looks interesting, but capturing all the prediction requests in BigQuery could get costly. I'd prefer a more targeted approach like C.
upvoted 0 times
...
Sheldon
2 years ago
Option C seems the most straightforward and minimizes disruption to the production environment. I like how it gradually ramps up the new model's traffic to monitor performance.
upvoted 0 times
Eun
2 years ago
Definitely. It's a smart approach to minimize disruption while testing the new model.
upvoted 0 times
...
Kirk
2 years ago
I think monitoring end-user metrics is key to ensuring the new model is performing well.
upvoted 0 times
...
Lavera
2 years ago
I agree. It's important to monitor performance before fully deploying the new model.
upvoted 0 times
...
Oretha
2 years ago
I agree. It's important to monitor the end-user metrics and gradually increase the traffic to the new model to ensure it performs well.
upvoted 0 times
...
Latricia
2 years ago
Option C seems like a good choice. It allows for gradual testing of the new model.
upvoted 0 times
...
Deangelo
2 years ago
Option C seems like the best approach. It allows for gradual testing of the new model without causing too much disruption.
upvoted 0 times
...
...

Save Cancel