Free Preparation Discussions
MENU
Databricks Certified Data Engineer Professional Exam - Topic 1 Question 2 Discussion
Topic #: 1
A data engineer wants to join a stream of advertisement impressions (when an ad was shown) with another stream of user clicks on advertisements to correlate when impression led to monitizable clicks.
Which solution would improve the performance?
A)
B)
C)
D)
When joining a stream of advertisement impressions with a stream of user clicks, you want to minimize the state that you need to maintain for the join. Option A suggests using a left outer join with the condition that clickTime == impressionTime, which is suitable for correlating events that occur at the exact same time. However, in a real-world scenario, you would likely need some leeway to account for the delay between an impression and a possible click. It's important to design the join condition and the window of time considered to optimize performance while still capturing the relevant user interactions. In this case, having the watermark can help with state management and avoid state growing unbounded by discarding old state data that's unlikely to match with new data.
Erick
3 months agoFiliberto
3 months agoLuann
3 months agoIlda
4 months agoLauran
4 months agoAliza
4 months agoJustine
4 months agoCristy
4 months agoRicki
5 months agoBuffy
5 months agoFanny
5 months agoHershel
5 months agoTora
5 months agoLouann
5 months agoYan
5 months agoVashti
5 months agoMargot
5 months agoBrittni
2 years agoNana
2 years agoWerner
2 years agoNana
2 years agoWerner
2 years agoJuliana
2 years agoWerner
2 years agoRebeca
2 years agoQuiana
2 years ago