Discuss Databricks Databricks Certified Data Engineer Associate Exam Topic 3 Question 50

A data analyst has a series of queries in a SQL program. The data analyst wants this program to run every day. They only want the final query in the program to run on Sundays. They ask for help from the data engineering team to complete this task.

Which of the following approaches could be used by the data engineering team to complete this task?

AThey could submit a feature request with Databricks to add this functionality.

BThey could wrap the queries using PySpark and use Python's control flow system to determine when to run the final query.

CThey could only run the entire program on Sundays.

DThey could automatically restrict access to the source table in the final query so that it is only accessible on Sundays.

EThey could redesign the data model to separate the data used in the final query into a new table.

Suggested Answer: B

This approach would allow the data engineering team to use the existing SQL program and add some logic to control the execution of the final query based on the day of the week. They could use thedatetimemodule in Python to get the current date and check if it is a Sunday. If so, they could run the final query, otherwise they could skip it. This way, they could schedule the program to run every day without changing the data model or the source table.Reference:PySpark SQL Module,Python datetime Module,Databricks Jobs

Get Premium Databricks Certified Data Engineer Associate Questions as Interactive Web-Based Practice Test or PDF Real Exam Format Detailed Explanations Instant Access Get Your Practice Test

33 comments · newest first

Rossana 10 months ago

I agree, option E could work but might be unnecessary.

Lashanda 10 months ago

Option D seems overly complicated for this task.

Hoa 10 months ago

Surprised they can't just schedule the final query separately!

Sunshine 10 months ago

I think option C is too limiting.

Krissy 10 months ago

Option B sounds like a solid plan!

Fanny 11 months ago

I vaguely remember something about separating data into different tables, so option E might be a valid approach, but it feels like overkill for just one query.

Jose 11 months ago

Option A seems like a long shot; I don't think submitting a feature request is practical for this situation.

Lorrine 11 months ago

I think option C is too limiting since they want the program to run daily, but I can't recall if there's a way to schedule specific queries differently.

Galen 11 months ago

I remember something about using control flow in Python, so option B sounds familiar, but I'm not entirely sure how it would work with SQL.

Oretha 12 months ago

I'm leaning towards option B as well. Using Python's control flow to determine when to run the final query seems like a flexible and efficient way to handle this requirement.

Yuonne 12 months ago

I think option E, redesigning the data model to separate the data used in the final query into a new table, could be a good long-term solution. It would make the program more modular and easier to maintain.

Barrett 12 months ago

Hmm, I'm not sure I fully understand the requirements here. Is there a way to automatically restrict access to the source table in the final query so that it's only accessible on Sundays, as mentioned in option D? That could be an interesting approach to explore.

Gilma 1 year ago

I'm a bit confused by this question. It's not clear to me why we can't just run the entire program on Sundays, as suggested in option C. That seems like the simplest solution.

Linwood 1 year ago

This seems like a straightforward problem. I think the best approach would be to use Python's control flow system to determine when to run the final query, as described in option B.

Lizbeth 1 year ago

Choosing option A would be like asking the janitor to fix the plumbing. Databricks is for data processing, not scheduling tasks.

Adell 1 year ago

Wow, option D is really creative. Restricting access to the source table is a clever way to achieve the desired result. I wonder if it would work in a real-world scenario, though.

Maynard 1 year ago

↳ replying to Adell

User 2: Yeah, it's definitely a unique approach. I'm curious to see if it would actually work seamlessly.

Fredric 1 year ago

↳ replying to Adell

User 1: Option D does sound interesting. I wonder how effective it would be in practice.

Reuben 1 year ago

Option C is just lazy. If the client wants the final query to run only on Sundays, we should find a way to make that happen without running the entire program on Sundays.

Pearline 1 year ago

↳ replying to Reuben

A: Exactly, we need to find a solution that meets the client's requirements without being lazy.

Freeman 1 year ago

↳ replying to Reuben

B: I agree. That way we can ensure the final query runs only on Sundays without running the entire program.

Nathalie 1 year ago

↳ replying to Reuben

A: I think option B is the best approach. We can use Python's control flow system to determine when to run the final query.

Erinn 1 year ago

I would go with option E. Redesigning the data model to separate the data used in the final query seems like a more robust and maintainable approach.

Alberto 1 year ago

↳ replying to Erinn

I agree, redesigning the data model could definitely help in the long run.

Larae 1 year ago

↳ replying to Erinn

Option E sounds like a good idea. It would make the process more efficient.

Wenona 1 year ago

I prefer option E, redesigning the data model seems like a more permanent solution.

Josefa 1 year ago

I agree with Cherilyn, using PySpark and Python's control flow system gives more flexibility.

Cherilyn 1 year ago

I think option B is the best approach.

Hermila 1 year ago

Option B makes the most sense to me. Using Python's control flow to determine when to run the final query is a flexible and efficient solution.

Lenna 1 year ago

↳ replying to Hermila

Let's go ahead and implement this solution then. It seems like the best option for our needs.

Barney 1 year ago

↳ replying to Hermila

I think it would make the process more efficient and easier to manage in the long run.

Loise 1 year ago

↳ replying to Hermila

It's definitely a more dynamic approach compared to just running the entire program on Sundays.

Lenna 1 year ago

↳ replying to Hermila

I agree, using PySpark and Python's control flow system gives us more control over when to run the final query.

Databricks Certified Data Engineer Associate Exam - Topic 3 Question 50 Discussion