Certified Associate Developer for Apache Spark Exam

Exam Certified Associate Developer for Apache Spark All QuestionsBrowse all questions from this exam

Question 84

The code block shown below contains an error. The code block intended to create a single-column DataFrame from Scala List years which is made up of integers. Identify the error.

Code block:

spark.createDataset(years)

The years list should be wrapped in another list like List(years) to make clear that it is a column rather than a row.

The data type is not specified – the second argument to createDataset should be IntegerType.

There is no operation createDataset – the createDataFrame operation should be used instead.

The result of the above is a Dataset rather than a DataFrame – the toDF operation must be called at the end.

The column name must be specified as the second argument to createDataset.

Correct Answer: C

To create a DataFrame from a Scala List of integers, the correct operation is createDataFrame, not createDataset. The code spark.createDataset(years) would not work because createDataset is used to create a Dataset, not a DataFrame. Therefore, the proper operation to use in this context is createDataFrame.

Discussion

zozoshankyOption: C

C is the answer

deadbeef38Option: D

Since this is a scala question, the correct syntax would be : spark.createDataset(years).toDF("year") but that isn't one of the options

Sowwy1Option: A

Official Databricks tests (where answer is A) Question 44 Which of the following code blocks creates a single-column DataFrame from Scala Listyears which is made up of integers? A. spark.createDataset(years).toDF B. spark.createDataFrame(years, IntegerType) C. spark.createDataset(years) D. spark.DataFrame(years, IntegerType) E. spark.createDataFrame(years)

Sowwy1

Hence I'll go for D.

tangerine141Option: C

C. There is no operation createDataset – the createDataFrame operation should be used instead. The correct method to create a DataFrame in Spark using Scala is createDataFrame, not createDataset. The correct syntax would be: scala Copy code val df = spark.createDataFrame(years.map(Tuple1.apply)).toDF("columnName") This assumes that years is a List of integers, and the resulting DataFrame will have a single column named "columnName".