Databricks Certified Data Engineer Professional Exam - Topic 3 Question 21 Discussion

Actual exam question for Databricks's Databricks Certified Data Engineer Professional exam

Question #: 21
Topic #: 3

[All Databricks Certified Data Engineer Professional Questions]

A data engineer needs to capture pipeline settings from an existing in the workspace, and use them to create and version a JSON file to create a new pipeline.

Which command should the data engineer enter in a web terminal configured with the Databricks CLI?

AUse the get command to capture the settings for the existing pipeline; remove the pipeline_id and rename the pipeline; use this in a create command

BStop the existing pipeline; use the returned settings in a reset command

CUse the alone command to create a copy of an existing pipeline; use the get JSON command to get the pipeline definition; save this to git

DUse list pipelines to get the specs for all pipelines; get the pipeline spec from the return results parse and use this to create a pipeline

Show Suggested Answer

Suggested Answer: A

The Databricks CLI provides a way to automate interactions with Databricks services. When dealing with pipelines, you can use the databricks pipelines get --pipeline-id command to capture the settings of an existing pipeline in JSON format. This JSON can then be modified by removing the pipeline_id to prevent conflicts and renaming the pipeline to create a new pipeline. The modified JSON file can then be used with the databricks pipelines create command to create a new pipeline with those settings.

Databricks Documentation on CLI for Pipelines: Databricks CLI - Pipelines

by Candida at Sep 16, 2024, 05:35 PM

Limited Time Offer

25%

Off

Get Premium Databricks Certified Data Engineer Professional Questions as Interactive Web-Based Practice Test or PDF

Contribute your Thoughts:

Submit Cancel

Wynell

3 months ago

Not sure about option D, seems a bit complicated.

upvoted 0 times

...

Helene

3 months ago

Wait, can you really just rename the pipeline like that?

upvoted 0 times

...

Michel

3 months ago

Definitely A, it’s the most straightforward.

upvoted 0 times

...

Viola

4 months ago

Nah, I’d go with option C instead.

upvoted 0 times

...

Judy

4 months ago

I think option A is the right way to go!

upvoted 0 times

...

Tatum

4 months ago

I thought we could use the alone command to create a copy, but that doesn't sound right now. I should have reviewed the JSON commands more thoroughly.

upvoted 0 times

...

Nana

4 months ago

I feel like option D might be the right choice since it mentions getting specs for all pipelines, but I’m a bit confused about how to parse the results.

upvoted 0 times

...

Jerry

4 months ago

I remember practicing a similar question where we had to retrieve pipeline settings, but I can't recall if we used a reset command or something else.

upvoted 0 times

...

Leslie

5 months ago

I think we need to use the get command to capture the settings, but I'm not sure if we have to rename the pipeline or just modify it directly.

upvoted 0 times

...

Tu

5 months ago

Option D looks interesting, but I'm not sure how reliable it would be to parse the list of pipelines to get the specs. Seems like there could be some edge cases there. I'd probably stick with A or C to be on the safe side.

upvoted 0 times

...

Lyda

5 months ago

I think option C is the way to go. Using the "alone" command to create a copy of the existing pipeline, then getting the JSON definition and saving it to git seems like a good way to version the pipeline settings.

upvoted 0 times

...

Cecily

5 months ago

Hmm, I'm a bit confused by the question. Do we need to stop the existing pipeline first before creating a new one? Option B mentions stopping the existing pipeline, but I'm not sure if that's necessary.

upvoted 0 times

...

Reid

5 months ago

This seems straightforward - I'd go with option A. Capture the settings for the existing pipeline, remove the pipeline ID, and use that to create a new pipeline.

upvoted 0 times

...

Lashawna

5 months ago

I think one advantage is that they let different projects use different versions of the same libraries without conflicts.

upvoted 0 times

...

Hershel

1 year ago

D) that's a clever approach, but it seems a bit overkill. Why not just use the get command like in A)? Less steps, you know?

upvoted 0 times

Lorrine

1 year ago

C) I think A) is the way to go. It's simpler and more efficient.

upvoted 0 times

...

Tawanna

1 year ago

A) Stop the existing pipeline; use the returned settings in a reset command

upvoted 0 times

...

Wilda

1 year ago

D) Yeah, I see your point. A) does seem more straightforward and less complicated.

upvoted 0 times

...

Jesus

1 year ago

A) Use the get command to capture the settings for the existing pipeline; remove the pipeline_id and rename the pipeline; use this in a create command

upvoted 0 times

...

Robt

1 year ago

B) stopping the existing pipeline and resetting it? That's not what the question is asking for. We need to create a new pipeline, not modify the existing one.

upvoted 0 times

...

Jenise

1 year ago

C) is interesting, but I don't think creating a copy of an existing pipeline is what the question is asking for. We need to create a new pipeline based on the existing one.

upvoted 0 times

...

Darrel

1 year ago

A) seems like the right approach to me. Capturing the settings and creating a new versioned pipeline sounds like the task at hand.

upvoted 0 times

Kaitlyn

1 year ago

D) Use list pipelines to get the specs for all pipelines; get the pipeline spec from the return results parse and use this to create a pipeline.

upvoted 0 times

...

Theodora

1 year ago

B) Stop the existing pipeline; use the returned settings in a reset command.

upvoted 0 times

...

Stefania

1 year ago

A) seems like the right approach to me. Capturing the settings and creating a new versioned pipeline sounds like the task at hand.

upvoted 0 times

...

Kayleigh

1 year ago

I'm feeling option A is the way to go. Can't go wrong with a good old get and create command combo.

upvoted 0 times

...

Matilda

1 year ago

I'm not sure, but I think option C could also work by creating a copy of an existing pipeline.

upvoted 0 times

...

Margot

1 year ago

Haha, option C is like the data engineer is trying to clone a pipeline. Gotta love those 'alone' commands!

upvoted 0 times

...

Carmela

1 year ago

Option B is weird. Why would the data engineer need to stop the existing pipeline? That doesn't seem necessary for this task.

upvoted 0 times

Sean

1 year ago

User3: Option D seems like the right approach, using the list pipelines command to get the specs.

upvoted 0 times

...

Gladis

1 year ago

User2: I agree, it's better to focus on capturing the settings and creating the JSON file.

upvoted 0 times

...

Thaddeus

1 year ago

Option B does seem unnecessary. It's not mentioned in the question.

upvoted 0 times

...

Josephine

1 year ago

I agree with Meghan, using list pipelines to get the specs for all pipelines makes sense.

upvoted 0 times

...

Marcos

1 year ago

Option D doesn't make sense to me. Why would the data engineer need to list all pipelines and parse the specs when they just need to capture the settings for a specific pipeline?

upvoted 0 times

Galen

1 year ago

User 4: Option D does seem a bit unnecessary for this task, listing all pipelines and parsing specs is overkill.

upvoted 0 times

...

Charisse

1 year ago

User 3: Option C also sounds like a good option, creating a copy of the existing pipeline and saving the JSON to git.

upvoted 0 times

...

Elza

1 year ago

User 2: I agree, using the get command to capture the settings for the existing pipeline makes sense.

upvoted 0 times

...

Jolanda

1 year ago

Option A seems like the most straightforward approach.

upvoted 0 times

...

Meghan

1 year ago

I think the correct command is D.

upvoted 0 times

...

Lynette

1 year ago

I think option A is the correct answer. The data engineer needs to capture the pipeline settings, remove the pipeline ID, and use them to create a new pipeline.

upvoted 0 times