Exam Certified Data Engineer Professional All QuestionsBrowse all questions from this exam
Question 137

Which configuration parameter directly affects the size of a spark-partition upon ingestion of data into Spark?

    Correct Answer: A

    The configuration parameter 'spark.sql.files.maxPartitionBytes' directly affects the size of a spark-partition upon ingestion of data into Spark. This parameter specifies the maximum number of bytes to be packed into a single partition when reading files, thereby controlling the partition size directly.

Discussion
vexor3Option: A

A is correct