Discuss Databricks Databricks Machine Learning Associate Exam Topic 2 Question 20

An organization is developing a feature repository and is electing to one-hot encode all categorical feature variables. A data scientist suggests that the categorical feature variables should not be one-hot encoded within the feature repository.

Which of the following explanations justifies this suggestion?

AOne-hot encoding is a potentially problematic categorical variable strategy for some machine learning algorithms.

BOne-hot encoding is dependent on the target variable's values which differ for each apaplication.

COne-hot encoding is computationally intensive and should only be performed on small samples of training sets for individual machine learning problems.

DOne-hot encoding is not a common strategy for representing categorical feature variables numerically.

Suggested Answer: A

In Spark ML, a transformer is an algorithm that can transform one DataFrame into another DataFrame. It takes a DataFrame as input and produces a new DataFrame as output. This transformation can involve adding new columns, modifying existing ones, or applying feature transformations. Examples of transformers in Spark MLlib include feature transformers like StringIndexer, VectorAssembler, and StandardScaler.

Databricks documentation on transformers: Transformers in Spark ML

Get Premium Databricks Machine Learning Associate Questions as Interactive Web-Based Practice Test or PDF Real Exam Format Detailed Explanations Instant Access Get Your Practice Test

31 comments · newest first

Marshall 10 months ago

I thought one-hot was the go-to for categorical variables?

Ricarda 11 months ago

It’s not uncommon, but definitely depends on the context.

Dorthy 11 months ago

Wait, is one-hot encoding really that bad?

Alex 11 months ago

Totally agree, it can lead to high dimensionality issues!

Asuncion 11 months ago

One-hot encoding can be tricky for some algorithms.

Tamar 12 months ago

I’m a bit confused about option D; I thought one-hot encoding was actually a common method for handling categorical variables.

Herman 12 months ago

I feel like we practiced a question similar to this, and I think one-hot encoding can be computationally heavy, which might relate to option C.

Yuonne 12 months ago

I think option A makes sense because some algorithms, like decision trees, might not need one-hot encoding at all.

Lelia 1 year ago

I remember discussing one-hot encoding in class, but I’m not sure if it’s always the best choice for every algorithm.

Jesusita 1 year ago

I'm a little confused by this question. The options don't seem to clearly explain why we shouldn't one-hot encode the categorical variables. I'll have to review my notes on feature engineering to make sure I understand the tradeoffs here.

Flo 1 year ago

Okay, let me see. I think the key here is that the data scientist is suggesting we shouldn't one-hot encode the categorical variables in the feature repository. That makes me think option A is the best explanation - one-hot encoding can be problematic for some algorithms.

Leeann 1 year ago

Hmm, I'm a bit unsure here. I know one-hot encoding is a common way to handle categorical variables, but the question is suggesting we shouldn't do it. I'll have to think this through carefully.

Alida 1 year ago

I'm pretty confident about this one. I think the answer is A - one-hot encoding can be problematic for some machine learning algorithms.

Emily 1 year ago

B) Yep, that makes the most sense. The target variable can change, so one-hot encoding shouldn't be in the feature repo.

Felice 1 year ago

Ha! 'Not a common strategy', that's a funny way to put it. I wonder what the 'uncommon' strategies are.

Galen 1 year ago

↳ replying to Felice

User 4: Yeah, it's not supported by most machine learning libraries either.

Layla 1 year ago

↳ replying to Felice

User 3: It's computationally intensive and should only be used on small samples of training sets.

Tony 1 year ago

↳ replying to Felice

User 2: I agree, it can be problematic for some machine learning algorithms.

Hermila 1 year ago

↳ replying to Felice

User 1: One-hot encoding is not a common strategy for representing categorical variables numerically.

India 1 year ago

C) Computationally intensive, huh? I guess one-hot encoding can be a bit heavy for the training set, so it's better to leave it for individual problems.

Jettie 1 year ago

E) Ah, I see. Some machine learning algorithms may not play well with one-hot encoding, so it's a good idea to avoid it in the feature repository.

Stephanie 1 year ago

↳ replying to Jettie

C) One-hot encoding is computationally intensive and should only be performed on small samples of training sets for individual machine learning problems.

Maurine 1 year ago

↳ replying to Jettie

E) That's a good point. It's important to consider how the target variable values can impact the effectiveness of one-hot encoding.

Gaston 1 year ago

↳ replying to Jettie

B) One-hot encoding is dependent on the target variable's values which differ for each application.

Glenna 1 year ago

B) Hmm, that makes sense. The target variable can vary across different applications, so one-hot encoding shouldn't be done at the feature repository level.

Chaya 1 year ago

↳ replying to Glenna

E) One-hot encoding is a potentially problematic categorical variable strategy for some machine learning algorithms.

Mitsue 1 year ago

↳ replying to Glenna

C) One-hot encoding is computationally intensive and should only be performed on small samples of training sets for individual machine learning problems.

Fernanda 1 year ago

↳ replying to Glenna

A) One-hot encoding is dependent on the target variable's values which differ for each application.

Joaquin 2 years ago

I disagree. One-hot encoding is necessary for certain algorithms.

Shawna 2 years ago

I agree with Arlette. It can be computationally intensive.

Arlette 2 years ago

I think one-hot encoding is not always the best approach.

Databricks Machine Learning Associate Exam - Topic 2 Question 20 Discussion