Google Professional Data Engineer Exam - Topic 2 Question 11 Discussion

Actual exam question for Google's Professional Data Engineer exam

Question #: 11
Topic #: 2

[All Professional Data Engineer Questions]

You operate an IoT pipeline built around Apache Kafka that normally receives around 5000 messages per second. You want to use Google Cloud Platform to create an alert as soon as the moving average over 1 hour drops below 4000 messages per second. What should you do?

AConsume the stream of data in Cloud Dataflow using Kafka IO. Set a sliding time window of 1 hour every 5 minutes. Compute the average when the window closes, and send an alert if the average is less than 4000 messages.

BConsume the stream of data in Cloud Dataflow using Kafka IO. Set a fixed time window of 1 hour. Compute the average when the window closes, and send an alert if the average is less than 4000 messages.

CUse Kafka Connect to link your Kafka message queue to Cloud Pub/Sub. Use a Cloud Dataflow template to write your messages from Cloud Pub/Sub to Cloud Bigtable. Use Cloud Scheduler to run a script every hour that counts the number of rows created in Cloud Bigtable in the last hour. If that number falls below 4000, send an alert.

DUse Kafka Connect to link your Kafka message queue to Cloud Pub/Sub. Use a Cloud Dataflow template to write your messages from Cloud Pub/Sub to BigQuery. Use Cloud Scheduler to run a script every five minutes that counts the number of rows created in BigQuery in the last hour. If that number falls below 4000, send an alert.

Show Suggested Answer

Suggested Answer: C

by Lonny at May 08, 2022, 12:03 PM

Limited Time Offer

25%

Off

Get Premium Professional Data Engineer Questions as Interactive Web-Based Practice Test or PDF

Contribute your Thoughts:

Submit Cancel

Mariann

4 months ago

Whoa, I didn't know you could link Kafka to Cloud Pub/Sub like that!

upvoted 0 times

...

Jackie

4 months ago

I disagree, C is too complicated for this task.

upvoted 0 times

...

Alida

4 months ago

Wait, why not just use D? Sounds easier!

upvoted 0 times

...

Barrie

4 months ago

I think B is more straightforward, though.

upvoted 0 times

...

Gerald

5 months ago

A seems like the best option for real-time alerts!

upvoted 0 times

...

Tasia

5 months ago

I’m a bit confused about the differences between options C and D. They both use Cloud Pub/Sub, but I can't recall which one is more efficient for this scenario.

upvoted 0 times

...

Arlene

5 months ago

I feel like option C could work too, but it seems more complicated with the extra steps of using Cloud Pub/Sub and Cloud Bigtable.

upvoted 0 times

...

Stanford

5 months ago

I remember we discussed sliding vs. fixed time windows in class. I think option A might be better for real-time alerts, but I'm not entirely sure.

upvoted 0 times

...

Elvera

5 months ago

I think I practiced a question similar to this where we had to compute averages over time. I’m leaning towards option B, but I’m not confident about the fixed window.

upvoted 0 times

...

Merissa

5 months ago

The edition concept sounds like it has to do with how the data is stored and managed in SAP MDG, financials. I'll need to review my notes on the different storage modes to answer this correctly.

upvoted 0 times

...

Selma

5 months ago

I'm not entirely sure about this one. I'll need to review my notes on bandwidth management techniques to see if any of these options ring a bell.

upvoted 0 times

...

Jackie

5 months ago

I feel like the NTP mismatch could also be a possibility, but I'm not entirely sure how to check if they're set to the same source.

upvoted 0 times

...