Free Preparation Discussions
MENU
Databricks Certified Associate Developer for Apache Spark 3.5 Exam - Topic 3 Question 14 Discussion
Topic #: 3
35 of 55.
A data engineer is building a Structured Streaming pipeline and wants it to recover from failures or intentional shutdowns by continuing where it left off.
How can this be achieved?
In Structured Streaming, checkpoints store state information (offsets, progress, and metadata) needed to resume a stream after a failure or restart.
Correct usage:
Set the checkpointLocation option when writing the streaming output:
streaming_df.writeStream
.format('delta')
.option('checkpointLocation', '/path/to/checkpoint/dir')
.start('/path/to/output')
Spark uses this checkpoint directory to recover progress automatically and maintain exactly-once semantics.
Why the other options are incorrect:
A/D: recoveryLocation is not a valid Spark configuration option.
B: Checkpointing must be configured in writeStream, not during readStream.
PySpark Structured Streaming Guide --- Checkpointing and recovery.
Databricks Exam Guide (June 2025): Section ''Structured Streaming'' --- explains checkpointing and fault-tolerant streaming recovery.
Dorthy
1 day agoGerardo
6 days agoNakisha
12 days agoDeja
17 days agoHannah
22 days agoAshlyn
27 days agoLeota
1 month agoBritt
1 month agoLanie
1 month agoDenise
2 months agoFanny
2 months agoFiliberto
2 months agoOretha
2 months agoWillodean
2 months agoEva
2 months agoJoni
3 months agoRozella
4 months agoTomas
4 months agoMarg
5 months ago