U.S. Independence Day Deal! Unlock 25% OFF Today – Limited-Time Offer - Ends In 00:00:00 Coupon code: SAVE25
Welcome to Pass4Success

- Free Preparation Discussions

Databricks Machine Learning Associate Exam - Topic 1 Question 40 Discussion

A data scientist has a Spark DataFrame spark_df. They want to create a new Spark DataFrame that contains only the rows from spark_df where the value in column price is greater than 0.Which of the following code blocks will accomplish this task?
B) spark_df.filter(col('price') > 0)
A) spark_df[spark_df['price'] > 0]
C) SELECT * FROM spark_df WHERE price > 0
D) spark_df.loc[spark_df['price'] > 0,:]
E) spark_df.loc[:,spark_df['price'] > 0]

Databricks Machine Learning Associate Exam - Topic 1 Question 40 Discussion

Actual exam question for Databricks's Databricks Machine Learning Associate exam
Question #: 40
Topic #: 1
[All Databricks Machine Learning Associate Questions]

A data scientist has a Spark DataFrame spark_df. They want to create a new Spark DataFrame that contains only the rows from spark_df where the value in column price is greater than 0.

Which of the following code blocks will accomplish this task?

Show Suggested Answer Hide Answer
Suggested Answer: B

To filter rows in a Spark DataFrame based on a condition, you use the filter method along with a column condition. The correct syntax in PySpark to accomplish this task is spark_df.filter(col('price') > 0), which filters the DataFrame to include only those rows where the value in the 'price' column is greater than 0. The col function is used to specify column-based operations. The other options provided either do not use correct Spark DataFrame syntax or are intended for different types of data manipulation frameworks like pandas. Reference:

PySpark DataFrame API documentation (Filtering DataFrames).


Contribute your Thoughts:

0/2000 characters
Lennie
1 day ago
Definitely going with B, it’s the standard method.
upvoted 0 times
...
Kati
2 months ago
I think A looks right too, but not sure.
upvoted 0 times
...
Laine
2 months ago
Option B is the correct way to filter in Spark!
upvoted 0 times
...
Jin
3 months ago
I think option A is incorrect because it looks more like Pandas syntax, not Spark.
upvoted 0 times
...
Blondell
3 months ago
I feel like I've seen something similar before, but I can't remember if `loc` is even applicable in Spark.
upvoted 0 times
...
Tom
3 months ago
I think option B looks right because I recall using `filter` for conditions in Spark DataFrames.
upvoted 0 times
...
Luther
3 months ago
I remember practicing filtering DataFrames, but I'm not sure if it's `filter` or `where` in Spark.
upvoted 0 times
...

Save Cancel