Deal of The Day! Hurry Up, Grab the Special Discount - Save 25% - Ends In 00:00:00 Coupon code: SAVE25
Welcome to Pass4Success

- Free Preparation Discussions

Databricks Certified Associate Developer for Apache Spark 3.5 Exam - Topic 7 Question 19 Discussion

A Spark developer is building an app to monitor task performance. They need to track the maximum task processing time per worker node and consolidate it on the driver for analysis.Which technique should be used?
A) Use an RDD action like reduce() to compute the maximum time
B) Use an accumulator to record the maximum time on the driver
C) Broadcast a variable to share the maximum time among workers
D) Configure the Spark UI to automatically collect maximum times

Databricks Certified Associate Developer for Apache Spark 3.5 Exam - Topic 7 Question 19 Discussion

Actual exam question for Databricks's Databricks Certified Associate Developer for Apache Spark 3.5 exam
Question #: 19
Topic #: 7
[All Databricks Certified Associate Developer for Apache Spark 3.5 Questions]

A Spark developer is building an app to monitor task performance. They need to track the maximum task processing time per worker node and consolidate it on the driver for analysis.

Which technique should be used?

Show Suggested Answer Hide Answer
Suggested Answer: A

The correct way to aggregate information (e.g., max value) from distributed workers back to the driver is using RDD actions such as reduce() or aggregate().

From the documentation:

''To perform global aggregations on distributed data, actions like reduce() are commonly used to collect summaries such as min/max/avg.''

Accumulators (Option B) do not support max operations directly and are not intended for such analytics.

Broadcast (Option C) is used to send data to workers, not collect from them.

Spark UI (Option D) is a monitoring tool --- not an analytics collection interface.

Final Answer: A


Contribute your Thoughts:

0/2000 characters
Lera
4 days ago
Not sure about broadcasting, seems inefficient for this case.
upvoted 0 times
...
Florencia
10 days ago
Accumulators are great for this kind of task!
upvoted 0 times
...
Luis
15 days ago
Wait, can the Spark UI really do that automatically?
upvoted 0 times
...
Glory
20 days ago
I disagree, reduce() could work too!
upvoted 0 times
...
Antione
25 days ago
I think option B is the best choice for tracking max time.
upvoted 0 times
...
Shonda
1 month ago
I don’t recall configuring the Spark UI for this purpose, but it seems like it wouldn’t give us the control we need over the data collection.
upvoted 0 times
...
Rosalia
1 month ago
Broadcasting a variable sounds familiar, but I feel like it might not actually consolidate the data on the driver as needed.
upvoted 0 times
...
Ashley
1 month ago
I remember practicing a similar question where we had to track metrics, and I think using an RDD action like reduce() might not be the best for this.
upvoted 0 times
...
Jody
2 months ago
I think using an accumulator could be the right choice here, but I'm not entirely sure how it works in this context.
upvoted 0 times
...

Save Cancel