A Data Engineer needs to load JSON output from some software into Snowflake using Snowpipe.
Which recommendations apply to this scenario? (Choose three.)
A Data Engineer needs to load JSON output from some software into Snowflake using Snowpipe.
Which recommendations apply to this scenario? (Choose three.)
When loading JSON data into Snowflake using Snowpipe, there are specific recommendations to follow for optimal performance and cost efficiency. First, it is recommended to ensure that data files are 100-250 MB or larger in size when compressed, helping to balance performance and cost (option B). Additionally, verifying each value of each unique element stores a single native data type (string or number) ensures data consistency and integrity (option D). Lastly, creating data files less than 100 MB and staging them in cloud storage at a frequency greater than once each minute might seem counterintuitive, but it helps to keep the overhead manageable and avoids issues with latency and queue management in Snowpipe (option F). This combination of practices ensures effective and efficient use of Snowpipe for loading JSON data.
https://docs.snowflake.com/en/user-guide/data-load-considerations-prepare
Loading data files roughly 100-250 MB in size or larger reduces the overhead charge relative to the amount of total data loaded to the point where the overhead cost is immaterial. https://docs.snowflake.com/en/user-guide/data-load-considerations-prepare#label-snowpipe-file-size
A - incorrect as size too large; C - incorrect as not good practice for any data platform; F - Quote from Snowflake: Creating smaller data files and staging them in cloud storage more often than once per minute has the following disadvantages: - A reduction in latency between staging and loading the data cannot be guaranteed. - An overhead to manage files in the internal load queue is included in the utilization costs charged for Snowpipe. This overhead increases in relation to the number of files queued for loading.