A data engineer needs to apply custom logic to identify employees with more than 5 years of experience in array column employees in table stores. The custom logic should create a new column exp_employees that is an array of all of the employees with more than 5 years of experience for each row. In order to apply this custom logic at scale, the data engineer wants to use the FILTER higher-order function.
Which of the following code blocks successfully completes this task?
The FILTER higher-order function is used to filter elements of an array based on a specified condition. In this context, we need to filter the 'employees' array to retain only those employees with more than 5 years of experience. The appropriate code block to achieve this uses the FILTER function correctly by specifying the array 'employees' and the lambda function to apply the condition 'i -> i.years_exp > 5'. This will create a new column 'exp_employees' with the filtered array for each row. Therefore, the code block in option A is correct.
A is correct.
https://docs.databricks.com/en/sql/language-manual/functions/filter.html
A is correct
A is correct
B & E incorrect : source is employees not exp_employees D incorrect : does not use FILTER higher-order function) C incorrect : syntax errror A : correct by elimination & based on https://docs.databricks.com/en/sql/language-manual/functions/filter.html#examples