MENU
You need to develop the task logic for a new job in Lakeflow Jobs that processes telemetry data.
Each task must contain only the appropriate logic for its step in the pipeline. The solution must support the planned changes and meet the data ingestion and processing requirements.
What should you do?
CORRECT ANSWE R: D - Create separate tasks for ingestion, cleansing, and curation.
According to Microsoft Learn, Lakeflow Jobs (formerly Databricks Workflows) supports multi-task pipelines where each task encapsulates a single, well-defined step. The official documentation states that best practice is to decompose complex pipelines into discrete tasks --- ingestion, cleansing, and curation --- so that each task contains only the logic appropriate for that stage. This approach aligns with the Contoso planned change to 'implement scalable data pipeline orchestration' and the requirement to 'define a clear execution order and dependencies.' Option A is incorrect because combining all logic in one notebook violates the single-responsibility principle and makes retry/recovery difficult. Option B is incorrect because duplicating identical logic across tasks wastes resources and defeats the purpose of a modular pipeline. Option C is incorrect because a single SQL MERGE task cannot cleanly separate the ingestion, cleansing, and curation concerns required.