MENU
A data engineer and data analyst are working together on a data pipeline. The data engineer is working on the raw, bronze, and silver layers of the pipeline using Python, and the data analyst is working on the gold layer of the pipeline using SQL The raw source of the pipeline is a streaming input. They now want to migrate their pipeline to use Delta Live Tables.
Which change will need to be made to the pipeline when migrating to Delta Live Tables?
Suggested Answer: A
Get Premium Databricks Certified Data Engineer Associate Questions
as Interactive Web-Based Practice Test or PDF
Real Exam Format
Detailed Explanations
Instant Access
Get Your Practice Test
When migrating to Delta Live Tables (DLT) with a data pipeline that involves different programming languages across various data layers, the migration does not require unifying the pipeline into a single language. Delta Live Tables support multi-language pipelines, allowing data engineers and data analysts to work in their preferred languages, such as Python for data engineering tasks (raw, bronze, and silver layers) and SQL for data analytics tasks (gold layer). This capability is particularly beneficial in collaborative settings and leverages the strengths of each language for different stages of data processing.
Reference: Databricks documentation on Delta Live Tables: Delta Live Tables Guide
47 comments · newest first
DA
Dahlia
10 months ago
I heard they can still use streaming sources, so D seems wrong too.
GI
Gianna
11 months ago
No way it needs to be entirely in Python! That’s not how Delta Live Tables work.
RI
Rickie
11 months ago
Wait, are they really saying it has to be all SQL? That sounds off.
FR
Francisca
11 months ago
Definitely agree with that! Mixing languages is a plus.
TA
Tanesha
11 months ago
I think the pipeline can have different notebook sources in SQL & Python.
GL
Glenna
12 months ago
I thought Delta Live Tables could work with both batch and streaming sources, so I’m leaning towards A as well.
FE
Felix
12 months ago
I practiced a similar question, and I think the streaming input is still valid with Delta Live Tables, so option D doesn't seem right.
JO
Jolanda
12 months ago
I'm not entirely sure, but I feel like the pipeline doesn't have to be written entirely in one language. That seems too restrictive.
AS
Ashton
1 year ago
I remember we discussed how Delta Live Tables can support both SQL and Python, so I think option A might be correct.
AI
Aileen
1 year ago
I've worked with Delta Live Tables before, and I know it supports both SQL and Python. My guess is that option A is the correct answer, since the pipeline can continue to use the different notebook sources.
ME
Meghan
1 year ago
Okay, let's see. The pipeline is currently using a streaming input, and Delta Live Tables is being introduced. I'm guessing the change needed has something to do with that transition from streaming to batch.
QU
Quentin
1 year ago
I'm a bit confused on this one. The question mentions using both SQL and Python, but the answer choices seem to imply we need to choose one or the other. I'll need to think this through carefully.
BE
Bettina
1 year ago
Hmm, this seems straightforward. I think the key is understanding how Delta Live Tables works and how it differs from the current pipeline setup.
AR
Arlen
1 year ago
Hmm, this one seems tricky. I'll need to think carefully about the TCP flags and how to detect a Null Scan attempt.
DU
Dudley
2 years ago
Option B, easy peasy. If it's good enough for the data engineer, it's good enough for the data analyst. Time to brush up on that SQL, my friend. Oh, and don't forget the backup sunglasses for when the code inevitably blinds you with its brilliance.
CO
Cordelia
2 years ago
Option C, no doubt! Gotta keep that Python love alive, am I right? I mean, who needs SQL when you can just write the whole pipeline in Python? It's like the '90s all over again!
JO
Jolanda
2 years ago
We can still use SQL for that layer, no problem.
BR
Bronwyn
2 years ago
But what about the SQL part for the gold layer?
WI
Willie
2 years ago
Agreed, Python is so much more flexible and powerful.
YA
Yasuko
2 years ago
I think we should stick with Python for the pipeline.
KI
King
2 years ago
Whoa, this is a tricky one! I'm going to have to go with option A. Mixing and matching notebook sources in SQL and Python sounds like a recipe for a programmer's paradise. Or a nightmare, depending on your perspective.
ME
Merissa
2 years ago
Hmm, I think option D is the way to go. Using a batch source instead of a streaming one for Delta Live Tables seems like the logical choice. After all, who needs real-time when you can have delayed data, am I right?
LA
Lavonna
2 years ago
Data engineer: Let's make the change and see how it improves the pipeline.
AD
Adelina
2 years ago
Data analyst: Yeah, it might be more efficient that way.
VO
Vonda
2 years ago
Data engineer: I agree, switching to a batch source for Delta Live Tables makes sense.
KI
Kiley
2 years ago
Actually, I think the pipeline can still have different notebook sources in SQL & Python even with Delta Live Tables.
LO
Louvenia
2 years ago
I bet the data analyst is secretly hoping for option B, just so they can lord their SQL skills over the data engineer. Classic power move!
MA
Marvel
2 years ago
B: I wonder if we'll have to switch to writing the pipeline entirely in SQL.
LA
Lashaun
2 years ago
A: Yeah, that sounds right. It'll be a big change for us.
TE
Temeka
2 years ago
B: I think we'll need to use a batch source instead of a streaming source.
ME
Mee
2 years ago
A: We need to migrate our pipeline to Delta Live Tables.
BR
Brande
2 years ago
I agree. I believe the pipeline will need to be written entirely in SQL for Delta Live Tables.
GE
Gerardo
2 years ago
Wait, what? Option B wants me to write the entire pipeline in SQL? That's a hard pass. I'm sticking with Python, thanks.
KI
Kiley
2 years ago
I think we will need to make some changes to our pipeline when migrating to Delta Live Tables.
JE
Jenise
2 years ago
Option B? Really? Forcing the whole pipeline to be in SQL seems like a bit of a stretch. This is a tough one.
AZ
Azalee
2 years ago
Interesting dilemma. I wonder if they could keep the SQL and Python separation, but just wrap it all in Delta Live Tables. Guess we'll have to see what the experts say!
HA
Harley
2 years ago
Maybe we can find a way to integrate both Python and SQL into Delta Live Tables.
YE
Yen
2 years ago
But what about the SQL part of the pipeline? We can't just abandon that.
ER
Erick
2 years ago
I think we might need to rewrite the pipeline entirely in Python.
EL
Elvis
2 years ago
I'm not sure about that. The question says the data engineer is working with Python, so I think option C might be the right choice.
FR
Francesco
2 years ago
Let's make the necessary changes to migrate to Delta Live Tables.
JE
Jesus
2 years ago
That makes sense since the data engineer is already working with Python.
KI
Kimberely
2 years ago
I agree. I believe the pipeline will need to be written entirely in Python.
NA
Nada
2 years ago
I think we need to switch to using Delta Live Tables for our pipeline.
HO
Hortencia
2 years ago
Hmm, I think option D is the way to go. Migrating to Delta Live Tables likely requires using a batch source instead of a streaming one.
CH
Chau
2 years ago
Yeah, it's important to make sure the pipeline is compatible with Delta Live Tables.
EU
Eura
2 years ago
I agree, using a batch source would make the migration smoother.