Databricks Exam Databricks Certified Data Engineer Associate Topic 4 Question 37 Discussion

Actual exam question for Databricks's Databricks Certified Data Engineer Associate exam

Question #: 37
Topic #: 4

[All Databricks Certified Data Engineer Associate Questions]

A data engineer and data analyst are working together on a data pipeline. The data engineer is working on the raw, bronze, and silver layers of the pipeline using Python, and the data analyst is working on the gold layer of the pipeline using SQL The raw source of the pipeline is a streaming input. They now want to migrate their pipeline to use Delta Live Tables.

Which change will need to be made to the pipeline when migrating to Delta Live Tables?

AThe pipeline can have different notebook sources in SQL & Python.

BThe pipeline will need to be written entirely in SQL.

CThe pipeline will need to be written entirely in Python.

DThe pipeline will need to use a batch source in place of a streaming source.

Show Suggested Answer

Suggested Answer: A

When migrating to Delta Live Tables (DLT) with a data pipeline that involves different programming languages across various data layers, the migration does not require unifying the pipeline into a single language. Delta Live Tables support multi-language pipelines, allowing data engineers and data analysts to work in their preferred languages, such as Python for data engineering tasks (raw, bronze, and silver layers) and SQL for data analytics tasks (gold layer). This capability is particularly beneficial in collaborative settings and leverages the strengths of each language for different stages of data processing.

Reference: Databricks documentation on Delta Live Tables: Delta Live Tables Guide

by Paz at Sep 18, 2024, 07:28 PM

Limited Time Offer

25%

Off

Get Premium Databricks Certified Data Engineer Associate Questions as Interactive Web-Based Practice Test or PDF

Contribute your Thoughts:

Submit Cancel

Dudley

8 months ago

Option B, easy peasy. If it's good enough for the data engineer, it's good enough for the data analyst. Time to brush up on that SQL, my friend. Oh, and don't forget the backup sunglasses for when the code inevitably blinds you with its brilliance.

upvoted 0 times

...

Cordelia

9 months ago

Option C, no doubt! Gotta keep that Python love alive, am I right? I mean, who needs SQL when you can just write the whole pipeline in Python? It's like the '90s all over again!

upvoted 0 times

Jolanda

7 months ago

We can still use SQL for that layer, no problem.

upvoted 0 times

...

Bronwyn

8 months ago

But what about the SQL part for the gold layer?

upvoted 0 times

...

Willie

8 months ago

Agreed, Python is so much more flexible and powerful.

upvoted 0 times

...

Yasuko

8 months ago

I think we should stick with Python for the pipeline.

upvoted 0 times

...

King

9 months ago

Whoa, this is a tricky one! I'm going to have to go with option A. Mixing and matching notebook sources in SQL and Python sounds like a recipe for a programmer's paradise. Or a nightmare, depending on your perspective.

upvoted 0 times

...

Merissa

9 months ago

Hmm, I think option D is the way to go. Using a batch source instead of a streaming one for Delta Live Tables seems like the logical choice. After all, who needs real-time when you can have delayed data, am I right?

upvoted 0 times

Lavonna

8 months ago

Data engineer: Let's make the change and see how it improves the pipeline.

upvoted 0 times

...

Adelina

8 months ago

Data analyst: Yeah, it might be more efficient that way.

upvoted 0 times

...

Vonda

8 months ago

Data engineer: I agree, switching to a batch source for Delta Live Tables makes sense.

upvoted 0 times

...

Kiley

9 months ago

Actually, I think the pipeline can still have different notebook sources in SQL & Python even with Delta Live Tables.

upvoted 0 times

...

Louvenia

9 months ago

I bet the data analyst is secretly hoping for option B, just so they can lord their SQL skills over the data engineer. Classic power move!

upvoted 0 times

Marvel

8 months ago

B: I wonder if we'll have to switch to writing the pipeline entirely in SQL.

upvoted 0 times

...

Lashaun

8 months ago

A: Yeah, that sounds right. It'll be a big change for us.

upvoted 0 times

...

Temeka

9 months ago

B: I think we'll need to use a batch source instead of a streaming source.

upvoted 0 times

...

Mee

9 months ago

A: We need to migrate our pipeline to Delta Live Tables.

upvoted 0 times

...

Brande

9 months ago

I agree. I believe the pipeline will need to be written entirely in SQL for Delta Live Tables.

upvoted 0 times

...

Gerardo

9 months ago

Wait, what? Option B wants me to write the entire pipeline in SQL? That's a hard pass. I'm sticking with Python, thanks.

upvoted 0 times

...

Kiley

9 months ago

I think we will need to make some changes to our pipeline when migrating to Delta Live Tables.

upvoted 0 times

...

Jenise

9 months ago

Option B? Really? Forcing the whole pipeline to be in SQL seems like a bit of a stretch. This is a tough one.

upvoted 0 times

...

Azalee

9 months ago

Interesting dilemma. I wonder if they could keep the SQL and Python separation, but just wrap it all in Delta Live Tables. Guess we'll have to see what the experts say!

upvoted 0 times