MENU
A machine learning engineer would like to develop a linear regression model with Spark ML to predict the price of a hotel room. They are using the Spark DataFrame train_df to train the model.
The Spark DataFrame train_df has the following schema:
The machine learning engineer shares the following code block:
Which of the following changes does the machine learning engineer need to make to complete the task?
Suggested Answer: B
Get Premium Databricks Machine Learning Associate Questions
as Interactive Web-Based Practice Test or PDF
Real Exam Format
Detailed Explanations
Instant Access
Get Your Practice Test
In Spark ML, the linear regression model expects the feature column to be a vector type. However, if the features column in the DataFrame train_df is not already in this format (such as being a column of type UDT or a non-vectorized type), the engineer needs to convert it to a vector column using a transformer like VectorAssembler. This is a critical step in preparing the data for modeling as Spark ML models require input features to be combined into a single vector column.
Reference
29 comments · newest first
TA
Tamie
6 months ago
Splitting features could be useful, but I think vectorizing is key.
RO
Rozella
6 months ago
I agree with Roslyn. A Pipeline streamlines the process.
RO
Roslyn
6 months ago
No way, a Pipeline is essential for fitting the model properly.
ON
Onita
7 months ago
I’m not sure. Maybe they don’t need to make any changes?
GA
Gaynell
7 months ago
Yeah, that makes sense. The model needs the right format.
SY
Sylvie
7 months ago
I thought they could just use the existing features without changes.
IO
Iola
8 months ago
A Pipeline would definitely help streamline the process!
MA
Margurite
8 months ago
Wait, do they really need to split the features column? Seems unnecessary.
CH
Cherry
8 months ago
Totally agree, option B is the way to go!
PE
Pearlene
8 months ago
Option C seems too easy. There's gotta be a catch somewhere. I'd double-check that code if I were the engineer.
SH
Sharmaine
8 months ago
Haha, I bet the engineer is wishing they had a crystal ball to see the right answer. Good thing we have these practice exams to prepare!
JO
Jose
8 months ago
I think Option D is the way to go. Using a Pipeline makes the whole process a lot cleaner and more organized.
EL
Elenore
9 months ago
I think splitting the features into separate columns could be an option, but it seems like overkill for a linear regression model.
LE
Lenna
9 months ago
I feel like we had a practice question about transforming DataFrames, but I can't recall if it was specifically about calling the transform method.
LI
Lizette
9 months ago
I'm not entirely sure, but I think using a Pipeline might be necessary for organizing the steps in the model training process.
WH
Whitney
9 months ago
I remember we discussed the importance of converting the features column into a vector format for Spark ML. That seems crucial here.
BR
Brent
9 months ago
Splitting the features column into individual columns seems like overkill for a linear regression model. I think I'll stick with option B and make sure the features column is a vector. That should do the trick.
RY
Ryan
9 months ago
I'm a little unsure about the Pipeline part. Should I be using a Pipeline to fit the model? Option D is making me second-guess myself. I'll need to research that a bit more.
MA
Magda
10 months ago
Option B looks good to me. Gotta get those features into a vector format for the model to work properly.
GE
Gertude
10 months ago
Okay, let me think this through. The code block looks good, so I don't think I need to make any changes. I'll go with option C and see if that works.
LE
Lemuel
10 months ago
They need to convert the features column to be a vector.
LA
Lauran
10 months ago
I think they need to convert the features column to be a vector.
RO
Royal
11 months ago
I'm a bit confused here. The code block shows that the features column is already a vector, so I'm not sure if I need to do anything else. Maybe I should double-check the documentation to be sure.
KI
Kimbery
11 months ago
Hmm, this looks like a pretty straightforward linear regression problem. I'd start by making sure the features column is a vector, so I'd go with option B.
SH
Sherell
5 months ago
Option A could also be important after that, right?
PA
Paul
6 months ago
True, but I think the vector conversion is the first step.
CA
Cammy
6 months ago
Yeah, without a vector, it won't work properly.
AL
Aleta
6 months ago
I agree, option B seems essential for the model.
CR
Cristal
10 months ago
But what about using a Pipeline? Option D could be useful too.