Deal of The Day! Hurry Up, Grab the Special Discount - Save 25% - Ends In 00:00:00 Coupon code: SAVE25
Welcome to Pass4Success

- Free Preparation Discussions
Mail Us support@pass4success.com
Location US
Mob_nav_barsMENU

Databricks Certified Associate Developer for Apache Spark 3.5 Exam - Topic 4 Question 20 Discussion

A Spark DataFrame df is cached using the MEMORY_AND_DISK storage level, but the DataFrame is too large to fit entirely in memory.What is the likely behavior when Spark runs out of memory to store the DataFrame?
C) Spark will store as much data as possible in memory and spill the rest to disk when memory is full, continuing processing with performance overhead.
A) Spark duplicates the DataFrame in both memory and disk. If it doesn't fit in memory, the DataFrame is stored and retrieved from the disk entirely.
B) Spark splits the DataFrame evenly between memory and disk, ensuring balanced storage utilization.
D) Spark stores the frequently accessed rows in memory and less frequently accessed rows on disk, utilizing both resources to offer balanced performance.

Databricks Certified Associate Developer for Apache Spark 3.5 Exam - Topic 4 Question 20 Discussion

Actual exam question for Databricks's Databricks Certified Associate Developer for Apache Spark 3.5 exam
Question #: 20
Topic #: 4
[All Databricks Certified Associate Developer for Apache Spark 3.5 Questions]

A Spark DataFrame df is cached using the MEMORY_AND_DISK storage level, but the DataFrame is too large to fit entirely in memory.

What is the likely behavior when Spark runs out of memory to store the DataFrame?

Show Suggested Answer Hide Answer
Suggested Answer: C

When using the MEMORY_AND_DISK storage level, Spark attempts to cache as much of the DataFrame in memory as possible. If the DataFrame does not fit entirely in memory, Spark will store the remaining partitions on disk. This allows processing to continue, albeit with a performance overhead due to disk I/O.

As per the Spark documentation:

'MEMORY_AND_DISK: It stores partitions that do not fit in memory on disk and keeps the rest in memory. This can be useful when working with datasets that are larger than the available memory.'

--- Perficient Blogs: Spark - StorageLevel

This behavior ensures that Spark can handle datasets larger than the available memory by spilling excess data to disk, thus preventing job failures due to memory constraints.


by Val at Aug 12, 2026, 08:00 PM

Limited Time Offer

25%

Off

Get Premium Databricks Certified Associate Developer for Apache Spark 3.5 Questions as Interactive Web-Based Practice Test or PDF

Contribute your Thoughts:

0/2000 characters
Submit Cancel
Dacia
5 days ago
Definitely not A or B, those don't make sense in this context.
upvoted 0 times
...
Elvera
10 days ago
Wait, so it just spills over? That’s surprising!
upvoted 0 times
...
Honey
15 days ago
D sounds plausible, but I don't think that's how Spark handles it.
upvoted 0 times
...
Valda
20 days ago
I thought it would duplicate the DataFrame, but that's not how it works.
upvoted 0 times
...
Nettie
25 days ago
C is the right answer! It spills to disk when memory is full.
upvoted 0 times
...
Lennie
1 month ago
I remember that Spark tries to optimize performance, so C seems right. It makes sense that it would spill to disk when memory runs out.
upvoted 0 times
...
Sherman
1 month ago
I’m leaning towards A, but it doesn’t quite match what I recall about how Spark handles memory. I need to double-check the caching behavior.
upvoted 0 times
...
Beckie
1 month ago
I feel like I’ve seen a question like this before, and I think it was about how Spark manages memory. C sounds familiar, but D also seems plausible.
upvoted 0 times
...
Pa
2 months ago
I think the answer might be C, but I'm not entirely sure. I remember something about spilling data to disk when memory is full.
upvoted 0 times
...

Save Cancel