Certified Associate Developer for Apache Spark Exam

Exam Certified Associate Developer for Apache Spark All QuestionsBrowse all questions from this exam

Question 173

Which of the following code blocks returns a DataFrame containing a column month, an integer representation of the month from column openDate from DataFrame storesDF?

Note that column openDate is of type integer and represents a date in the UNIX epoch format — the number of seconds since midnight on January 1 st, 1970.

A sample of storesDF is displayed below:

storesDF.withColumn("month", getMonth(col("openDate")))

storesDF.withColumn("month", substr(col("openDate"), 4, 2))

(storesDF.withColumn("openDateFormat", col("openDate").cast("Date"))

.withColumn("month", month(col("openDateFormat"))))

(storesDF.withColumn("openTimestamp", col("openDate").cast("Timestamp"))

.withColumn("month", month(col("openTimestamp"))))

storesDF.withColumn("month", month(col("openDate")))

Correct Answer: D

The correct approach involves converting the UNIX epoch time (which is an integer) into a timestamp so that the month can be extracted from it. To do this, we need to first cast the integer column `openDate` to a timestamp and then extract the month from this timestamp. Hence, the code block that accomplishes this is `(storesDF.withColumn(

Discussion

Sowwy1Option: D

D. (storesDF.withColumn("openTimestamp", col("openDate").cast("Timestamp")) .withColumn("month", month(col("openTimestamp"))))