Independence Day Deal! Unlock 25% OFF Today – Limited-Time Offer - Ends In 00:00:00 Coupon code: SAVE25
Welcome to Pass4Success

- Free Preparation Discussions

Databricks Exam Databricks Certified Associate Developer for Apache Spark 3.0 Topic 1 Question 41 Discussion

Actual exam question for Databricks's Databricks Certified Associate Developer for Apache Spark 3.0 exam
Question #: 41
Topic #: 1
[All Databricks Certified Associate Developer for Apache Spark 3.0 Questions]

Which of the following code blocks immediately removes the previously cached DataFrame transactionsDf from memory and disk?

Show Suggested Answer Hide Answer
Suggested Answer: C

Correct code block:

transactionsDf.withColumn('cos', round(cos(degrees(transactionsDf.value)),2))

This Question: is especially confusing because col, 'cos' are so similar. Similar-looking answer options can also appear in the exam and, just like in this question, you need to pay attention to

the

details to identify what the correct answer option is.

The first answer option to throw out is the one that starts with withColumnRenamed: The Question: speaks specifically of adding a column. The withColumnRenamed operator only renames

an

existing column, however, so you cannot use it here.

Next, you will have to decide what should be in gap 2, the first argument of transactionsDf.withColumn(). Looking at the documentation (linked below), you can find out that the first argument of

withColumn actually needs to be a string with the name of the column to be added. So, any answer that includes col('cos') as the option for gap 2 can be disregarded.

This leaves you with two possible answers. The real difference between these two answers is where the cos and degree methods are, either in gaps 3 and 4, or vice-versa. From the QUESTION

NO: you

can find out that the new column should have 'the values in column value converted to degrees and having the cosine of those converted values taken'. This prescribes you a clear order of

operations: First, you convert values from column value to degrees and then you take the cosine of those values. So, the inner parenthesis (gap 4) should contain the degree method and then,

logically, gap 3 holds the cos method. This leaves you with just one possible correct answer.

More info: pyspark.sql.DataFrame.withColumn --- PySpark 3.1.2 documentation

Static notebook | Dynamic notebook: See test 3, Question: 49 (Databricks import instructions)


Contribute your Thoughts:

Rusty
2 months ago
Wait, 'persist()? Are you kidding me? That's the opposite of what we want to do! Oh, these exam questions, they really keep you on your toes, don't they?
upvoted 0 times
Billy
28 days ago
User 3: These exam questions are tricky, you have to pay attention to the details.
upvoted 0 times
...
Ayesha
29 days ago
User 2: Yeah, that's correct. 'persist()' would keep it in memory and disk.
upvoted 0 times
...
Shawn
29 days ago
User 3: That's right, 'unpersist' removes the DataFrame from memory and disk.
upvoted 0 times
...
Owen
1 months ago
User 2: I think the correct answer is 'transactionsDf.unpersist()'.
upvoted 0 times
...
Major
1 months ago
User 1: Yeah, 'persist' is definitely not what we want here.
upvoted 0 times
...
Howard
1 months ago
User 1: I think the answer is B) transactionsDf.unpersist()
upvoted 0 times
...
...
Elvis
2 months ago
Ooh, 'clearCache()' sounds promising, but I'm guessing that's just for removing it from memory, not disk. Gotta be thorough here, 'unpersist()' is the way to go.
upvoted 0 times
...
Aja
2 months ago
Hmm, 'del transactionsDf' might work, but I'm not sure if that actually removes it from disk. Better play it safe and go with the 'unpersist()' option.
upvoted 0 times
Kara
27 days ago
User 3: Better play it safe and go with the 'unpersist()' option.
upvoted 0 times
...
Keneth
28 days ago
User 2: I'm not sure if that actually removes it from disk.
upvoted 0 times
...
Catrice
1 months ago
User 1: I think 'del transactionsDf' might work.
upvoted 0 times
...
Lavonne
1 months ago
User3: Better play it safe and go with the 'unpersist()' option.
upvoted 0 times
...
Gabriele
2 months ago
User2: I'm not sure if that actually removes it from disk.
upvoted 0 times
...
Quentin
2 months ago
User1: I think 'del transactionsDf' might work.
upvoted 0 times
...
...
Burma
2 months ago
Alright, let's see here... B is the correct answer, 'unpersist()' is the way to go to remove that DataFrame from memory and disk.
upvoted 0 times
...
Jules
2 months ago
I'm not sure, but I think D) transactionsDf.clearCache() might also work.
upvoted 0 times
...
Nada
2 months ago
I agree with Brynn, unpersist() removes the DataFrame from memory and disk.
upvoted 0 times
...
Ruthann
2 months ago
Haha, what's the deal with option A? 'array_remove'? That's not even a thing in Spark, is it?
upvoted 0 times
Hayley
1 months ago
User2: I agree. Option B is the correct one to remove the cached DataFrame.
upvoted 0 times
...
Serina
2 months ago
User1: Yeah, option A doesn't make sense. It's not a valid function in Spark.
upvoted 0 times
...
...
Brynn
3 months ago
I think the answer is B) transactionsDf.unpersist().
upvoted 0 times
...

Save Cancel