Free Preparation Discussions
MENU
Databricks Certified Data Engineer Professional Exam - Topic 1 Question 2 Discussion
Topic #: 1
A data engineer wants to join a stream of advertisement impressions (when an ad was shown) with another stream of user clicks on advertisements to correlate when impression led to monitizable clicks.
Which solution would improve the performance?
A)
B)
C)
D)
When joining a stream of advertisement impressions with a stream of user clicks, you want to minimize the state that you need to maintain for the join. Option A suggests using a left outer join with the condition that clickTime == impressionTime, which is suitable for correlating events that occur at the exact same time. However, in a real-world scenario, you would likely need some leeway to account for the delay between an impression and a possible click. It's important to design the join condition and the window of time considered to optimize performance while still capturing the relevant user interactions. In this case, having the watermark can help with state management and avoid state growing unbounded by discarding old state data that's unlikely to match with new data.
Erick
9 months agoFiliberto
9 months agoLuann
9 months agoIlda
10 months agoLauran
10 months agoAliza
10 months agoJustine
10 months agoCristy
10 months agoRicki
11 months agoBuffy
11 months agoFanny
11 months agoHershel
11 months agoTora
11 months agoLouann
11 months agoYan
11 months agoVashti
11 months agoMargot
11 months agoBrittni
2 years agoNana
2 years agoWerner
2 years agoNana
2 years agoWerner
2 years agoJuliana
2 years agoWerner
2 years agoRebeca
2 years agoQuiana
2 years ago