MENU
A Data engineer wants to run unit's tests using common Python testing frameworks on python functions defined across several Databricks notebooks currently used in production.
How can the data engineer run unit tests against function that work with data in production?
Suggested Answer: A
Get Premium Databricks Certified Data Engineer Professional Questions
as Interactive Web-Based Practice Test or PDF
Real Exam Format
Detailed Explanations
Instant Access
Get Your Practice Test
The best practice for running unit tests on functions that interact with data is to use a dataset that closely mirrors the production data. This approach allows data engineers to validate the logic of their functions without the risk of affecting the actual production data. It's important to have a representative sample of production data to catch edge cases and ensure the functions will work correctly when used in a production environment.
Databricks Documentation on Testing: Testing and Validation of Data and Notebooks
35 comments · newest first
AD
Adelle
10 months ago
D is a good approach if you want to keep things organized.
JE
Jerlene
10 months ago
Wait, can you really run tests in production notebooks? That seems sketchy!
AM
Amie
10 months ago
I disagree, B could be more efficient with version control.
GL
Gladys
10 months ago
C sounds risky, mixing tests with production code isn't ideal.
BO
Boris
11 months ago
A is the safest option for testing without risking production data.
CA
Carline
11 months ago
I think importing unit test functions from a separate notebook is a good way to keep tests modular, but I wonder if it complicates the setup too much.
DE
Delisa
11 months ago
I feel like defining unit tests in the same notebook could lead to confusion, but I can't recall if that's a common practice or not.
RU
Ruby
11 months ago
I'm a bit unsure about the best approach, but I think using Files in Repos could help keep things organized, like we practiced in our last session.
AN
Antonio
12 months ago
I remember we discussed the importance of using non-production data for testing to avoid any issues, so I think option A makes sense.
EL
Eloisa
12 months ago
I'm a bit confused on the best way to approach this. Should I just define everything in the same notebook? Or is it better to separate the unit tests and functions? I'll need to review the options carefully.
DE
Desire
12 months ago
Okay, I've got a plan. I'll use non-production data that closely mirrors the real thing, and define the unit tests and functions in separate notebooks. That way I can test everything thoroughly without impacting the live system.
NI
Nieves
1 year ago
Hmm, I think defining the functions and unit tests in separate files or notebooks might be the way to go. That way I can keep everything organized and easily reusable.
IL
Ilene
1 year ago
This looks like a tricky one. I'm not sure if I should use production data or not for the unit tests. I'll have to think carefully about the pros and cons of each approach.
TA
Tamesha
2 years ago
Option D is the clear winner here. Separate the tests from the actual code - that's the professional way to do it. Unless, of course, you're a mad scientist who enjoys mixing it all together. *cue evil laughter*
VA
Vanna
1 year ago
Rolande: That's a valid point, but separating them can help with debugging and maintenance.
IZ
Izetta
1 year ago
User 3: I prefer having everything in one notebook for easier access.
RO
Rolande
1 year ago
User 2: Agreed, it's important to maintain clean and organized code.
JE
Jettie
1 year ago
User 1: Option D is definitely the way to go. Keep the tests separate from the code.
UN
Una
2 years ago
I think option C is the most practical solution.
TH
Thaddeus
2 years ago
Importing the unit test functions from a separate notebook is the way to go. Nice and modular, just like how I like my code.
JO
Jolene
2 years ago
I prefer option B, it's easier to manage.
HU
Hubert
2 years ago
I disagree, I believe option D is more efficient.
MA
Marci
2 years ago
Putting the unit tests in the same notebook as the functions? That just seems messy and hard to manage. Definitely going with option D.
CR
Cristen
1 year ago
Defining and unit testing functions using Files in Repos might provide a more organized way to manage the tests.
RE
Remedios
1 year ago
Running unit tests against non-production data that closely mirrors production could also be a good approach.
ED
Eden
2 years ago
I agree, putting unit tests in the same notebook can get messy. Option D seems like a better choice.
EV
Evan
2 years ago
I think option A is the best choice.
MA
Mayra
2 years ago
Defining functions in a repository and unit testing them there sounds like a good approach to me. Keeps things organized and maintainable.
GL
Glory
1 year ago
Defining functions in a repository and unit testing them there sounds like a good approach to me. Keeps things organized and maintainable.
AN
Andree
1 year ago
B) Define and unit test functions using Files in Repos
DI
Dion
1 year ago
A) Run unit tests against non-production data that closely mirrors production
SO
Sonia
2 years ago
Option A makes the most sense. Running tests against non-production data ensures we don't disrupt the actual production environment.
TR
Trevor
2 years ago
B) Define and unit test functions using Files in Repos
HE
Helga
2 years ago
That's a good point. We definitely don't want to mess with the production data.
CA
Carmela
2 years ago
A) Run unit tests against non-production data that closely mirrors production