MENU
A data scientist is working on a project that requires processing large amounts of structured data, performing SQL queries, and applying machine learning algorithms. The data scientist is considering using Apache Spark for this task.
Which combination of Apache Spark modules should the data scientist use in this scenario?
Options:
Comprehensive
To cover structured data processing, SQL querying, and machine learning in Apache Spark, the correct combination of components is:
Spark DataFrames: for structured data processing
Spark SQL: to execute SQL queries over structured data
MLlib: Spark's scalable machine learning library
This trio is designed for exactly this type of use case.
Why other options are incorrect:
A: GraphX is for graph processing --- not needed here.
B: Pandas API on Spark is useful, but MLlib is essential for ML, which this option omits.
C: Spark Streaming is legacy; GraphX is irrelevant here.
Currently there are no comments in this discussion, be the first to comment!