You have an Azure Databricks workspace that is enabled for Unity Catalog and contains a Delta table named Orders.
You load the Orders table into an Apache Spark DataFrame named df.
You need to create a DataFrame that excludes rows where the order amount is null.
Solution: You run the following expression.
df.dropna(subset=["order_amount"])
Does this meet the goal?
CORRECT ANSWE R: A - Yes.
According to Microsoft Learn on PySpark DataFrame operations, df.dropna(subset=['order_amount']) removes all rows from the DataFrame where the specified column (order_amount) contains a null value. The resulting DataFrame contains only rows where order_amount is not null, which directly meets the requirement to 'create a DataFrame that excludes rows where the order_amount is null.' The dropna() method (equivalent to DataFrame.na.drop()) is the idiomatic PySpark approach for removing rows with null values in specified columns. The subset parameter limits the null check to only the order_amount column, preserving rows where other columns may be null. This is the correct and recommended approach for null row exclusion in PySpark.
Currently there are no comments in this discussion, be the first to comment!