A company plans to store hundreds of files in an Azure Storage account and in Azure Data Lake Storage account. The files will be stored in the parquet format. A solution must be in place that would adopt the following requirements:
- Provide the ability to process the data every 5 hours
- Give the ability for interactive data analysis
- Give the ability to process data using solid-state drive caching
- Make use of Directed Acyclic Graph processing mechanisms
- Provide support for REST API calls for monitoring purposes
- Ensure support for Python and Integration with Microsoft Power BI
Which of the following would you consider for the solution?