Deal of The Day! Hurry Up, Grab the Special Discount - Save 25% - Ends In 00:00:00 Coupon code: SAVE25
Welcome to Pass4Success

- Free Preparation Discussions

Databricks Certified Associate Developer for Apache Spark 3.5 Exam - Topic 4 Question 21 Discussion

Posted by Aracelis on Sep 26, 2026 at 03:19 AM

46 of 55.

A data engineer is implementing a streaming pipeline with watermarking to handle late-arriving records.

The engineer has written the following code:

inputStream \

.withWatermark("event_time", "10 minutes") \

.groupBy(window("event_time", "15 minutes"))

What happens to data that arrives after the watermark threshold?

Suggested Answer: A

Watermarking in Structured Streaming defines how late a record can arrive based on event time before Spark discards it.

Behavior:

.withWatermark('event_time', '10 minutes')

This means Spark will keep state for 10 minutes beyond the maximum event time seen so far.

Any data arriving later than 10 minutes after the current watermark is ignored --- it will not be included in the aggregation or output.

Why the other options are incorrect:

B: Late data beyond the watermark threshold is not included.

C: Late data is not moved to a new window; it's simply dropped.

D: True for late data within the watermark threshold, not after it.


Spark Structured Streaming Guide --- withWatermark() behavior and late data handling.

Databricks Exam Guide (June 2025): Section ''Structured Streaming'' --- watermarking and state cleanup behavior.

===========

Get Premium Databricks Certified Associate Developer for Apache Spark 3.5 Questions as Interactive Web-Based Practice Test or PDF Real Exam Format Detailed Explanations Instant Access Get Your Practice Test
9 comments · newest first
JA
Janey 3 days ago
I thought late data could still be included, but not like that!
DE
Detra 8 days ago
D) sounds right too, late data within the threshold is processed.
RE
Regenia 14 days ago
Wait, what if the data is just a few seconds late?
IL
Ilene 19 days ago
Totally agree, that's how watermarking works!
YU
Yun 24 days ago
A) is correct, late data gets ignored after the watermark.
ME
Melvin 29 days ago
I feel like I might be mixing up the concepts here; I thought late data could still be processed if it was within the window, which makes me think of option C.
LI
Lizbeth 1 month ago
I practiced a similar question where late data was included if it was within the window, so I'm tempted to say option D is correct.
GR
Gracia 1 month ago
I think any data arriving after the watermark is ignored, so I would lean towards option A, but I need to double-check that.
NA
Naomi 1 month ago
I remember that watermarking is used to handle late data, but I'm not entirely sure how it interacts with the windowing.
0/2000 characters

Save Cancel