MENU
Your organization has a petabyte of application logs stored as Parquet files in Cloud Storage. You need to quickly perform a one-time SQL-based analysis of the files and join them to data that already resides in BigQuery. What should you do?
Suggested Answer: C
Get Premium Associate Data Practitioner Questions
as Interactive Web-Based Practice Test or PDF
Real Exam Format
Detailed Explanations
Instant Access
Get Your Practice Test
Creating external tables over the Parquet files in Cloud Storage allows you to perform SQL-based analysis and joins with data already in BigQuery without needing to load the files into BigQuery. This approach is efficient for a one-time analysis as it avoids the time and cost associated with loading large volumes of data into BigQuery. External tables provide seamless integration with Cloud Storage, enabling quick and cost-effective analysis of data stored in Parquet format.
31 comments · newest first
TI
Tiara
10 months ago
Wait, can you really do SQL joins with external tables? Sounds too good to be true!
NA
Nathan
10 months ago
I think B could work too, but it might be overkill for a one-time analysis.
TE
Terrilyn
11 months ago
A seems like too much setup for a quick job.
GE
Gerald
11 months ago
D is not ideal, loading a petabyte into BigQuery sounds like a nightmare.
SH
Shawnda
11 months ago
C is the easiest way to do this! Just create external tables.
FL
Fletcher
11 months ago
I recall that Cloud Data Fusion is useful for ETL processes, but for a quick SQL analysis, I wonder if option C is still the best approach.
AL
Alfred
11 months ago
I practiced a question similar to this, and I feel like using the bq load command in option D might be overkill for a one-time analysis.
CH
Cherelle
12 months ago
I'm not entirely sure, but I remember something about using Dataproc for big data processing. Maybe option A is the right choice?
JA
Janella
12 months ago
I think option C sounds familiar; creating external tables over the Parquet files seems like a straightforward way to analyze them without moving data.
GA
Garry
1 year ago
For a petabyte-scale dataset, I'd be hesitant to use the bq load command (option D). That could take a really long time to ingest all the data. I'm inclined to go with option C and create the external tables in BigQuery. Seems like the most efficient way to get the job done.
MA
Matilda
1 year ago
Option B with Cloud Data Fusion looks interesting, but I'm not as familiar with that service. I think I'll stick with the more traditional BigQuery approach and go with option C. The external table integration should be pretty straightforward.
GL
Glenn
1 year ago
I'm a bit unsure about this one. The question mentions a "one-time" analysis, so I'm not sure if creating a Dataproc cluster for a PySpark job is the best use of resources. I'm leaning towards option C or D, but I'll need to think it through a bit more.
AD
Adell
1 year ago
This seems like a straightforward data integration task, so I'll likely go with option C. Creating external tables in BigQuery to join the Parquet files from Cloud Storage seems like the most efficient approach.
RO
Rory
2 years ago
Option B seems like the way to go. Cloud Data Fusion makes it easy to connect multiple data sources and perform complex analyses.
BE
Bernardo
1 year ago
Let's go with option B then. It seems like the most efficient solution for this scenario.
SH
Shannan
1 year ago
It definitely sounds like the most straightforward way to quickly analyze the data.
ME
Melissia
1 year ago
I agree, using plugins to connect to the data sources and performing SQL join operations seems efficient.
LA
Laurel
1 year ago
I think option B is the best choice. Cloud Data Fusion can easily connect to both BigQuery and Cloud Storage.
MO
Monroe
2 years ago
Ha! Looks like we've got some 'Big Data' on our hands. I'd go with option D - keep it simple, stupid!
TH
Theresia
2 years ago
PySpark is overkill for a one-time analysis. Option C looks like the most straightforward approach here.
SU
Suzan
1 year ago
Yeah, using external tables to perform SQL joins seems like the simplest solution.
MA
Maybelle
1 year ago
I agree, Option C seems like the most efficient choice for this scenario.
AR
Ardella
2 years ago
I'm not a fan of external tables - they can be a bit of a pain to manage. I'd go with option D and just load the Parquet files directly into BigQuery.
IR
Irma
2 years ago
I think option D could work too, loading the files into BigQuery.
FI
Filiberto
2 years ago
I prefer option C, creating external tables over the files in Cloud Storage.
FA
Fairy
2 years ago
I agree, using Dataproc cluster with PySpark seems efficient.
DA
Daniel
2 years ago
Cloud Data Fusion seems like the easiest way to get this done. No need to write any code!
JO
Joni
1 year ago
C) Create external tables over the files in Cloud Storage, and perform SQL joins to tables in BigQuery to analyze the data.
PE
Peggy
1 year ago
B) Launch a Cloud Data Fusion environment, use plugins to connect to BigQuery and Cloud Storage, and use the SQL join operation to analyze the data.
SA
Sarah
2 years ago
A) Create a Dataproc cluster, and write a PySpark job to join the data from BigQuery to the files in Cloud Storage.
SO
Sommer
2 years ago
I think option A sounds like a good idea.