Cloudera Exam CCA175 Topic 1 Question 45 Discussion

Actual exam question for Cloudera's CCA175 exam

Question #: 45
Topic #: 1

[All CCA175 Questions]

Problem Scenario 34 : You have given a file named spark6/user.csv.

Data is given below:

user.csv

id,topic,hits

Rahul,scala,120

Nikita,spark,80

Mithun,spark,1

myself,cca175,180

Now write a Spark code in scala which will remove the header part and create RDD of values as below, for all rows. And also if id is myself" than filter out row.

Map(id -> om, topic -> scala, hits -> 120)

ASolution :
Step 1 : Create file in hdfs (We will do using Hue). However, you can first create in local filesystem and then upload it to hdfs.
Step 2 : Load user.csv file from hdfs and create PairRDDs val csv = sc.textFile('spark6/user.csv')
Step 3 : split and clean data
val headerAndRows = csv.map(line => line.split(',').map(_.trim))
Step 4 : Get header row
val header = headerAndRows.first
Step 5 : Filter out header (We need to check if the first val matches the first header name) val data = headerAndRows.filter(_(0) != header(O))
Step 6 : Splits to map (header/value pairs)
val maps = data.map(splits => header.zip(splits).toMap)
step 7: Filter out the user 'myself
val result = maps.filter(map => mapf'id') != 'myself')
Step 8 : Save the output as a Text file. result.saveAsTextFile('spark6/result.txt')

BSolution :
Step 1 : Create file in hdfs (We will do using Hue). However, you can first create in local filesystem and then upload it to hdfs.
Step 2 : Load user.csv file from hdfs and create PairRDDs val csv = sc.textFile('spark6/user.csv')
Step 3 : split and clean data
val headerAndRows = csv.map(line => line.split(',').map(_.trim))
Step 4 : Get header row
val header = headerAndRows.first
Step 5 : Splits to map (header/value pairs)
val maps = data.map(splits => header.zip(splits).toMap)
step 6: Filter out the user 'myself
val result = maps.filter(map => mapf'id') != 'myself')
Step 7 : Save the output as a Text file. result.saveAsTextFile('spark6/result.txt')

Show Suggested Answer

Suggested Answer: A

by Chantell at Jul 26, 2023, 03:13 AM

Limited Time Offer

25%

Off

Get Premium CCA175 Questions as Interactive Web-Based Practice Test or PDF

Contribute your Thoughts:

Submit Cancel

Currently there are no comments in this discussion, be the first to comment!