MENU
You have trained a DNN regressor with TensorFlow to predict housing prices using a set of predictive features. Your default precision is tf.float64, and you use a standard TensorFlow estimator;
estimator tf.estimator.DNNRegressor(
feature_columns[YOUR_LIST_OF_FEATURES],
hidden_units-[1024, 512, 256],
dropoutNone)
Your model performs well, but Just before deploying it to production, you discover that your current serving latency is 10ms @ 90 percentile and you currently serve on CPUs. Your production requirements expect a model latency of 8ms @ 90 percentile. You are willing to accept a small decrease in performance in order to reach the latency requirement Therefore your plan is to improve latency while evaluating how much the model's prediction decreases. What should you first try to quickly lower the serving latency?
Suggested Answer: D
Get Premium Professional Machine Learning Engineer Questions
as Interactive Web-Based Practice Test or PDF
Real Exam Format
Detailed Explanations
Instant Access
Get Your Practice Test
32 comments · newest first
HO
Hortencia
11 months ago
Definitely not increasing dropout in PREDICT mode!
AU
Audria
11 months ago
Wait, can quantization really make that big of a difference?
LA
Lashandra
11 months ago
Increasing dropout might hurt accuracy, not sure about that.
JA
Jacquelyne
11 months ago
I think quantization is a solid move too.
MA
Mattie
12 months ago
Switching to GPU should help with latency!
CA
Casie
12 months ago
I’m not confident about retraining with a higher dropout rate, it seems like it might hurt the model's performance too much.
RE
Reuben
1 year ago
Quantization sounds familiar; I think reducing precision to tf.float16 could help with latency without retraining.
EL
Elden
1 year ago
I think switching to GPU serving could really speed things up, but I wonder if it’s worth the cost.
LE
Lettie
1 year ago
I remember we discussed dropout rates in class, but I'm not sure increasing it in PREDICT mode would help with latency.
LO
Louvenia
1 year ago
Switching to GPU serving seems like the most straightforward option here. GPUs are generally better suited for fast inference, so that could be a good way to hit the latency target without sacrificing too much model performance.
FR
Freeman
1 year ago
Okay, let me think this through. Increasing the dropout rate and retraining the model could potentially reduce the model size and complexity, which might help with latency. But I'm not sure if that would be enough to meet the 8ms requirement.
MA
Margart
1 year ago
Hmm, this is a tricky one. I'm not sure if increasing the dropout rate in PREDICT mode would really help with the latency requirement. That seems like it might just hurt the model's performance.
TW
Twanna
1 year ago
Ah, I think the key here is the mention of quantization. Reducing the floating point precision to tf.float16 could definitely help with latency, and it sounds like the best option to try first based on the requirements.
SI
Simona
1 year ago
This seems like a straightforward conceptual question. I'll focus on the definition of a liability and think through each option carefully.
OL
Ollie
1 year ago
I think the answer is A. TARGETDDL seems like the parameter that would enable DDL replication for a Replicat.
LA
Lauran
1 year ago
Okay, I've got a strategy for this. The key is to remember that the units are tested first in a traditional way after integration, so that narrows down the options. I'll carefully consider each test environment and see which one makes the most sense.
TH
Theodora
1 year ago
Quantization is the way to go here. Reducing the precision to tf.float16 should give us a nice latency boost without sacrificing too much model performance. And hey, at least we're not trying to run it on a Commodore 64, right?
ZI
Zita
1 year ago
Haha, if I were the model, I'd be like 'Seriously? You want me to predict house prices in 8ms? What is this, the Flash's house?'
MA
Margarita
1 year ago
B) Increase the dropout rate to 0.8 and retrain your model.
KY
Kyoko
1 year ago
C) Switch from CPU to GPU serving
TE
Tenesha
1 year ago
A) Increase the dropout rate to 0.8 in_PREDICT mode by adjusting the TensorFlow Serving parameters
PA
Pamella
1 year ago
Switching to GPU serving is a good option, but it might be overkill for the 8ms latency requirement. I'd try the quantization approach first - reducing the precision to tf.float16 could give us a quicker win.
EM
Emilio
1 year ago
That's a good point. Let's try reducing the precision first and see if that helps with the latency.
RO
Rosio
1 year ago
D) Apply quantization to your SavedModel by reducing the floating point precision to tf.float16.
MA
Marilynn
1 year ago
C) Switch from CPU to GPU serving
CA
Catarina
2 years ago
Increasing the dropout rate to 0.8 in the _PREDICT mode seems like an interesting idea, but I'm not sure if that would actually improve the latency. Retraining the model with the higher dropout might work, but that could impact performance.
AI
Aide
1 year ago
B) Increase the dropout rate to 0.8 and retrain your model
JU
Junita
1 year ago
C) Switch from CPU to GPU serving
NU
Nu
1 year ago
A) Increase the dropout rate to 0.8 in _PREDICT mode by adjusting the TensorFlow Serving parameters
NG
Ngoc
2 years ago
But wouldn't applying quantization to the SavedModel also help reduce latency?
AL
Aliza
2 years ago
I agree with Loreen, that could help improve the latency.
LO
Loreen
2 years ago
I think we should try switching from CPU to GPU serving first.