You have an Azure Databricks workspace that is enabled for Unity Catalog and contains a Delta table named Orders.
You load the Orders table into an Apache Spark DataFrame named df.
You need to create a DataFrame that excludes rows where the order amount is null.
Solution: You run the following expression.
df.filter(df.order_amount.isNotNull())
Does this meet the goal?
The correct answer is A --- Yes.
df.filter(df.order_amount.isNotNull()) is the correct PySpark pattern for excluding null rows. The isNotNull() method is a Column method that returns True for every row where order_amount has a value and False for rows where it is null. Spark's filter keeps only the rows where the condition evaluates to True, producing a DataFrame with all null order_amount rows removed.
This works correctly because isNotNull() is explicitly null-aware --- unlike the != None comparison in Q52, it doesn't rely on Python equality semantics. Under the hood it maps to the SQL expression order_amount IS NOT NULL, which is unambiguous in both SQL and Spark.
Both df.filter(df.order_amount.isNotNull()) and df.dropna(subset=['order_amount']) produce identical results. The choice between them is stylistic --- isNotNull() reads more explicitly as a filter condition, while dropna is more compact when handling multiple columns.
You have an Azure Databricks workspace that is enabled for Unity Catalog and contains a Delta table named Orders.
You load the Orders table into an Apache Spark DataFrame named df.
You need to create a DataFrame that excludes rows where the order amount is null.
Solution: You run the following expression.
df.dropna(subset=["order_amount"])
Does this meet the goal?
CORRECT ANSWE R: A - Yes.
According to Microsoft Learn on PySpark DataFrame operations, df.dropna(subset=['order_amount']) removes all rows from the DataFrame where the specified column (order_amount) contains a null value. The resulting DataFrame contains only rows where order_amount is not null, which directly meets the requirement to 'create a DataFrame that excludes rows where the order_amount is null.' The dropna() method (equivalent to DataFrame.na.drop()) is the idiomatic PySpark approach for removing rows with null values in specified columns. The subset parameter limits the null check to only the order_amount column, preserving rows where other columns may be null. This is the correct and recommended approach for null row exclusion in PySpark.
You have an Azure Databricks workspace named Workspace1 that contains a lakehouse and is enabled for Unity Catalog.
You have a connection to a Microsoft SQL Server database named DB1.
You need to expose the schemas and tables of DB1 to meet the following requirements:
* The schemas and tables can be queried in Databricks.
* The schemas and tables appear alongside other Unity Catalog objects.
* The data is NOT copied into Databricks-managed storage.
Solution: You create a foreign catalog in Catalog Explorer.
Does this meet the goal?
CORRECT ANSWE R: A - Yes.
According to Microsoft Learn on Lakehouse Federation and Unity Catalog foreign catalogs, a foreign catalog is a Unity Catalog object that represents an external database (such as SQL Server) through a registered connection. Creating a foreign catalog in Catalog Explorer using an existing connection to DB1 exposes all schemas and tables from DB1 as queryable objects within Unity Catalog. These objects appear alongside native Unity Catalog objects in Catalog Explorer. Crucially, Lakehouse Federation queries the external database in place --- the data is never copied into Databricks-managed storage. This satisfies all three requirements: schemas and tables can be queried in Databricks, they appear alongside other Unity Catalog objects, and the data is not copied. The foreign catalog is created under the registered connection to DB1.
You have an Azure Databricks workspace that is enabled for Unity Catalog and contains a Delta table named Orders
You load the Orders table into an Apache Spark DataFrame named df.
You need to create a DataFrame that excludes rows where the order amount is null.
Solution: You run the following expression.
df-fillna(0, subset=['order_amount'])
Does this meet the goal?
CORRECT ANSWE R: B - No.
According to Microsoft Learn on PySpark DataFrame operations, df.fillna(0, subset=['order_amount']) replaces null values in the order_amount column with the integer 0. This does NOT exclude rows where order_amount is null --- it replaces the null with 0, meaning those rows are still included in the resulting DataFrame with 0 as the order_amount value. The requirement is to 'create a DataFrame that excludes rows where the order_amount is null' --- which means null rows must be removed (dropped), not filled. The correct operation to exclude null rows is df.dropna(subset=['order_amount']) or df.filter(df.order_amount.isNotNull()). fillna is used for data imputation (replacing nulls with a default value), which is a different operation from filtering out null rows.
You have an Azure Databricks workspace that uses serverless compute.
You need to ingest data by using Lakeflow Jobs. New records must be processed as soon as they become available.
Which type of job trigger should you use for the ingestion?
CORRECT ANSWE R: D - Continuous trigger.
According to Microsoft Learn on Lakeflow Jobs triggers, the Continuous trigger keeps the job running perpetually and immediately processes new records as they become available --- this directly satisfies 'New records must be processed as soon as they become available.' The Continuous trigger is specifically designed for near-real-time ingestion scenarios where latency must be minimized. Option A (Manual) requires a human to start each run, adding latency. Option B (File Arrival) triggers a job when new files land in a storage location, which is appropriate for file-based ingestion but introduces per-file trigger overhead rather than true continuous processing. Option C (Scheduled) runs at fixed intervals (e.g., hourly), introducing batch latency that is inconsistent with processing records 'as soon as they become available.'
Ali Chaudhry
1 day agoJae Hoang
15 days agoSergei Esposito
19 days agoSunita Pillai
25 days ago