You are a data engineer implementing a lambda architecture on Microsoft Azure. You use an open-source big data solution to collect, process, and maintain data.
The analytical data store performs poorly.
You must implement a solution that meets the following requirements:
✑ Provide data warehousing
✑ Reduce ongoing management activities
✑ Deliver SQL query responses in less than one second
You need to create an HDInsight cluster to meet the requirements.
Which type of cluster should you create?
D
Lambda Architecture with Azure:
Azure offers you a combination of following technologies to accelerate real-time big data analytics:
1. Azure Cosmos DB, a globally distributed and multi-model database service.
2. Apache Spark for Azure HDInsight, a processing framework that runs large-scale data analytics applications.
3. Azure Cosmos DB change feed, which streams new data to the batch layer for HDInsight to process.
4. The Spark to Azure Cosmos DB Connector
Note: Lambda architecture is a data-processing architecture designed to handle massive quantities of data by taking advantage of both batch processing and stream processing methods, and minimizing the latency involved in querying big data.
References:
https://sqlwithmanoj.com/2018/02/16/what-is-lambda-architecture-and-what-azure-offers-with-its-new-cosmos-db/
Could the write answer be D becase Spark has; 1) Interactive queries through spark-sql 2) Datawarehousing capabilities through Delta Lake (and also spark-sql creates in memory tables) 3) Less management because these are out-of-the-box features?
It also mentions SQL queries which is not Hive ( Interactive Query)
I think the answer should be A. Interactive Query. Here I am implementing Lambda architecture using a open source technology which can be Apache Spark and already in use. The prevailing issue here Analytical Processing is very slow , in another words queries are slow. So I created an HDInsight Cluster of type "Interactive Query" to support Analytical processing/ fast query access, data warehousing etc. We can use HiveQL on Interactive Query. Refer to https://docs.microsoft.com/en-us/azure/hdinsight/interactive-query/apache-interactive-query-get-started
Exam was updated on Nov 24, 2020. Didn't see too many questions from the test on ExamTopics...maybe 20-30% of the test questions. Suggest waiting a bit to take the test so that all the exam prep questions are updated. Exam definitely requires hands-on knowledge of the products. A lot of questions on CosmosDB consistency settings, encryption/security, monitoring/metrics.
Do you passed the exam after 24 Nov, is there any difference regarding these questions and the updated ones ? I mean if I prepared the exam with these version of a questions what is my chances to pass it ? thank you
hi to all, answer: https://azure.microsoft.com/pt-pt/blog/general-availability-of-hdinsight-interactive-query-blazing-fast-data-warehouse-style-queries-on-hyper-scale-data-2/ sub-second ! Summary This week at Ignite, we are pleased to announce general availability of Azure HDInsight Interactive Query. Backed by our enterprise-grade SLA, HDInsight Interactive Query brings sub-second speed to data warehouse style SQL queries to the hyper-scale data stored in commodity cloud storage. regards
How does Spark meet the requirements? Spark does not provide data warehousing by itself, it is not a data store.
Neither does any of the options, the last part of the question is key: Which type of cluster will you create, hence, Spark
Apache Spark is correct Answer !! https://docs.microsoft.com/en-us/azure/hdinsight/spark/apache-spark-overview
I also vote for Interactive Query as "An Interactive Query cluster is different from an Apache Hadoop cluster. It contains only the Hive service. Requirements: ✑ Provide data warehousing ( Yes) ✑ Reduce ongoing management activities (Not sure) ✑ Deliver SQL query responses in less than one second ( Yes)
Would suggest to use the original link from MS: https://docs.microsoft.com/en-us/azure/cosmos-db/lambda-architecture as better background documentation
The link you provided is redirected to What is Azure Synapse Link for Azure Cosmos DB (Preview)? wy?
https://docs.microsoft.com/en-us/azure/cosmos-db/lambda-architecture
The link you provided is redirected to What is Azure Synapse Link for Azure Cosmos DB (Preview)?
A - Interactive Query - "Deliver SQL query responses in less than one second" https://docs.microsoft.com/en-us/azure/hdinsight/interactive-query/apache-interactive-query-get-started
i think the logic to answer this question is: Lambda architecture: https://databricks.com/glossary/lambda-architecture Azure implementation: https://azure.microsoft.com/en-us/services/databricks/ Azure Databricks = Fast, easy, and collaborative Apache SparkTM based analytics service
https://azure.microsoft.com/en-in/blog/lambda-architecture-using-azure-cosmosdb-faster-performance-low-tco-low-devops/
D: Apache spark
Correct Answer in Spark because it is in memory
https://docs.microsoft.com/en-us/azure/cosmos-db/lambda-architectur
correct
D is correct based on Data warehousing requirement.