MENU
You want to migrate an Apache Spark 3 batch job from on-premises to Google Cloud. You need to minimally change the job so that the job reads from Cloud Storage and writes the result to BigQuery. Your job is optimized for Spark, where each executor has 8 vCPU and 16 GB memory, and you want to be able to choose similar settings. You want to minimize installation and management effort to run your job. What should you do?
Suggested Answer: A
Get Premium Professional Data Engineer Questions
as Interactive Web-Based Practice Test or PDF
Real Exam Format
Detailed Explanations
Instant Access
Get Your Practice Test
13 comments · newest first
SH
Shalon
11 months ago
Wait, can you really run Spark jobs on Serverless? Sounds interesting!
AL
Alethea
11 months ago
D? Really? That sounds like too much work.
MA
Maxima
11 months ago
C seems overkill for this task, right?
JE
Jerry
11 months ago
I think B is better for less management hassle.
MI
Mi
12 months ago
A Dataproc cluster is a solid choice for Spark jobs.
RA
Raylene
12 months ago
I’m leaning towards option D with Compute Engine, but I recall that it might involve more installation and management than using Dataproc.
LA
Laura
1 year ago
I practiced a similar question where Kubernetes was mentioned, but I feel like it might require more setup than I want for this migration.
UL
Ula
1 year ago
I think I read somewhere that Dataproc Serverless can help minimize management effort, which makes option B appealing, but I need to double-check the performance aspects.
ER
Erasmo
1 year ago
I remember that Dataproc is designed for running Spark jobs, so option A seems like a good fit, but I'm not sure if it's the most efficient choice.
NI
Nieves
1 year ago
I'm a little confused by the options here. Executing the job in a new Dataproc cluster (A) seems like it could work, but then we'd have to manage the cluster ourselves. And running it on a Compute Engine VM (D) doesn't seem to take advantage of the managed services that Google Cloud offers. I think I'm leaning towards the Dataproc Serverless option (B), but I'll need to double-check the details.
AR
Armando
1 year ago
Okay, I got this. The question is asking us to migrate the Spark job to Google Cloud, and it wants us to minimize installation and management effort. Based on that, I'd say the Dataproc Serverless option (B) is the way to go. It should let us run the Spark job with similar resource settings without having to manage the cluster ourselves.
NA
Natalie
1 year ago
Hmm, I'm a bit unsure about this one. There are a few options presented, and I'm not sure which one would be the best fit. I'll need to think through the tradeoffs of each approach.
JA
Jamika
1 year ago
This looks like a straightforward Spark migration question. I think the key is to find the Google Cloud service that can most closely match the on-premises Spark setup with minimal changes.