Discuss Databricks Databricks Certified Associate Developer For Apache Spark 3.5 Exam Topic 3 Question 17

A DataFrame df has columns name, age, and salary. The developer needs to sort the DataFrame by age in ascending order and salary in descending order.

Which code snippet meets the requirement of the developer?

Adf.orderBy(col('age').asc(), col('salary').asc()).show()

Bdf.sort('age', 'salary', ascending=[True, True]).show()

Cdf.sort('age', 'salary', ascending=[False, True]).show()

Ddf.orderBy('age', 'salary', ascending=[True, False]).show()

Suggested Answer: D

To sort a PySpark DataFrame by multiple columns with mixed sort directions, the correct usage is:

python

CopyEdit

df.orderBy('age', 'salary', ascending=[True, False])

age will be sorted in ascending order

salary will be sorted in descending order

The orderBy() and sort() methods in PySpark accept a list of booleans to specify the sort direction for each column.

Documentation Reference: PySpark API - DataFrame.orderBy

12 comments · newest first

Cammy 2 months ago

B doesn't sort salary correctly. D is the right choice!

Lemuel 2 months ago

I feel confused. Why not B? It seems simpler.

Rebecka 3 months ago

I think option D is correct. It sorts age ascending and salary descending.

Amber 3 months ago

Not sure about D, seems too simple for complex sorting.

Florinda 3 months ago

Wait, can you really sort like that in one line?

Cordie 3 months ago

D makes total sense, I agree with that choice!

Cordelia 3 months ago

I think B is wrong, it sorts both in ascending order.

Celestina 3 months ago

Option D is the correct one! Sorting by age asc and salary desc.

Angelo 4 months ago

I feel like option A is close, but I’m pretty sure we need to sort salary in descending order, which it doesn’t do.

Keneth 4 months ago

I practiced something similar last week, but I can't recall if we used "ascending" as a list or just as a single argument.

Janna 4 months ago

I think option D looks familiar; it seems like the right way to specify different sort orders for each column.

Erick 4 months ago

I remember sorting DataFrames, but I’m not sure if the orderBy function uses ascending as a parameter.

Databricks Certified Associate Developer for Apache Spark 3.5 Exam - Topic 3 Question 17 Discussion