Certified Associate Developer for Apache Spark Exam

Exam Certified Associate Developer for Apache Spark All QuestionsBrowse all questions from this exam

Question 88

The code block shown below contains an error. The code block intended to return a DataFrame containing a column dayOfYear, an integer representation of the day of the year from column openDate from DataFrame storesDF. Identify the error.

Note that column openDate is of type integer and represents a date in the UNIX epoch format – the number of seconds since midnight on January 1st, 1970.

A sample of storesDF is displayed below:

Code block:

storesDF.withColumn("dayOfYear", dayofyear(col("openDate")))

The dayofyear() operation cannot extract the day of year from a column of type integer – column openDate must first be converted to type Timestamp.

The dayofyear() operation takes a quoted column name rather than a Column object as its first argument – the first argument should be "openDate".

The dayofyear() operation cannot extract the day of year from a column of type integer – column openDate must first be converted to type Date.

The dayofyear() operation is not applicable in a withColumn() call – the newColumn() operation must be used instead.

There is no dayofyear() operation – the day of year number must be extracted using substring utilities.

Correct Answer: A

The dayofyear() operation requires the input column to be of type Timestamp. Given that the openDate column in storesDF is of type integer which represents the number of seconds since the UNIX epoch, this column must first be converted to a Timestamp type before the dayofyear() function can accurately extract the day of the year.

Discussion

tangerine141Option: A

A is correct

newusernameOption: A

A is correct

kaisa1234

But question 50 is the same question and states that the code block with an error is the correct

newusername

the answer there is wrong. I posted the correct one.