MENU
A machine learning engineering team wants to build a continuous pipeline for data preparation of a machine learning application. The team would like the data to be fully processed and made ready for inference in a series of equal-sized batches.
Which of the following tools can be used to provide this type of continuous processing?
Suggested Answer: C
Get Premium Databricks Machine Learning Professional Questions
as Interactive Web-Based Practice Test or PDF
Real Exam Format
Detailed Explanations
Instant Access
Get Your Practice Test
41 comments · newest first
VI
Virgina
10 months ago
Totally with you on Structured Streaming! It's the best option here.
AS
Ashlee
11 months ago
Wait, can AutoML even handle continuous data prep? Sounds off.
OM
Omega
11 months ago
Spark UDFs are more for batch processing, not continuous.
DO
Dolores
11 months ago
I think Delta Lake could work too, right?
UL
Ulysses
11 months ago
Definitely Structured Streaming for continuous processing!
SA
Sanda
12 months ago
AutoML seems off for this question; I don't think it handles the continuous pipeline aspect we're looking for.
PA
Paulina
12 months ago
Delta Lake sounds familiar, but I can't recall if it specifically supports continuous data preparation.
WI
Willodean
12 months ago
I think Structured Streaming is the right choice here. We practiced a similar question about streaming data in our last session.
FR
Frederick
1 year ago
I remember we discussed Spark UDFs in class, but I'm not sure if they are meant for continuous processing like this.
CH
Charolette
1 year ago
MLflow could be useful for managing the overall machine learning workflow, but I don't think it directly addresses the continuous data processing requirement. I'll need to think about how MLflow might fit into a broader solution for this problem.
CH
Chaya
1 year ago
Delta Lake could be a good fit for the data storage and management aspect of this problem, but I'm not sure if it provides the continuous processing capabilities on its own. I'll need to investigate how Delta Lake integrates with other tools like Structured Streaming.
SH
Shanda
1 year ago
AutoML seems like an interesting choice, but I'm not sure if it would provide the level of control and customization needed for a continuous data processing pipeline. I'll have to research the capabilities of AutoML more.
KA
Katie
1 year ago
Hmm, I'm not too familiar with Structured Streaming. I wonder if Spark UDFs could also be a viable solution for this problem. I'll need to look into the capabilities of both options.
VI
Viva
1 year ago
This looks like a question about building a continuous data processing pipeline. I think Structured Streaming would be a good option to consider since it's designed for this type of use case.
TA
Tayna
1 year ago
Ah, I remember this now. The vgextend command is used to add new physical volumes to an existing volume group. That's the answer I'm going with.
VE
Venita
1 year ago
Yeah, I feel like this situation was in one of our practice case studies. The key is to find the right balance between following regulations and the urgency of the council's request.
CH
Chery
1 year ago
I'm pretty sure the output should include both the numbers and their corresponding string representations, but I'm confused about how they'd be ordered.
LI
Ling
1 year ago
Hey, I heard MLflow is the new hotness for machine learning ops. Maybe they can just throw some emojis at the data and it'll magically get processed.
MA
Maryann
1 year ago
D: I've heard Delta Lake is good for managing large amounts of data efficiently.
MA
Marge
1 year ago
C: Spark UDFs might also be useful for data processing.
BE
Bernardine
1 year ago
B: Yeah, MLflow is great for managing the machine learning lifecycle.
GE
Gertude
1 year ago
A: I think MLflow could definitely help with that.
MA
Mari
1 year ago
Spark UDFs? That's just for extending Spark's functionality, not for continuous data processing. I'm with the others - Structured Streaming is the way to go.
MA
Marci
1 year ago
AutoML? Really? That's for automating the machine learning model development process, not data preprocessing. I'd say Structured Streaming is the clear winner here.
JU
Julie
1 year ago
Spark UDFs might work well for custom data processing functions within the pipeline.
HE
Helaine
1 year ago
I think MLflow could also be useful for tracking and managing the machine learning pipeline.
TI
Tijuana
1 year ago
I agree, Structured Streaming is the best choice for continuous processing.
SA
Sabina
1 year ago
Delta Lake is great for reliable data lakes, but maybe not the best fit for this specific task.
EM
Emerson
1 year ago
MLflow could also be useful for tracking experiments and managing the machine learning lifecycle.
KE
Kerrie
1 year ago
I agree, Structured Streaming is the best choice for continuous processing.
JA
Janey
1 year ago
Hmm, I'm not sure about Structured Streaming. Isn't that more for stream processing? I feel like Delta Lake might be a better fit since it can handle batch processing as well.
DO
Dortha
1 year ago
Let's go with Delta Lake for the continuous processing pipeline.
AN
Annmarie
1 year ago
I'm not sure about Structured Streaming either, but Delta Lake seems like a versatile option.
LU
Lucina
1 year ago
I agree, Delta Lake can handle both batch and stream processing.
SU
Susana
1 year ago
I think Delta Lake would be a good choice for continuous processing.
LO
Loren
2 years ago
I think Structured Streaming is the right choice here. It's specifically designed for continuous, real-time data processing, which is exactly what the team needs for their machine learning pipeline.
JU
Judy
1 year ago
I think MLflow could also be a good option for managing the machine learning pipeline.
GE
Geraldine
1 year ago
I agree, Structured Streaming is perfect for continuous processing.
BE
Beth
2 years ago
I personally prefer using MLflow for managing the machine learning pipeline.
ST
Starr
2 years ago
I agree with Denae, Structured Streaming is a good choice for processing data in batches.
DE
Denae
2 years ago
I think Structured Streaming can be used for continuous processing.