A data engineer is building a data orchestration workflow. The data engineer plans to use a hybrid model that includes some on-premises resources and some resources that are in the cloud. The data engineer wants to prioritize portability and open source resources.
Which service should the data engineer use in both the on-premises environment and the cloud-based environment?
Amazon Managed Workflows for Apache Airflow (Amazon MWAA) is the appropriate choice for a data engineer looking to prioritize portability and open-source resources in a hybrid model. Amazon MWAA is a managed service for Apache Airflow, an open-source workflow automation tool. Apache Airflow can be deployed both on-premises and in the cloud, which aligns with the data engineer's need to use the same orchestration tool across different environments. This ensures consistency and portability, meeting the primary requirements outlined in the question.
Answer is C
Airflow can orchestrate workflows that involve both on-premises and cloud resources, making it ideal for hybrid models.
C. Amazon Managed Workflows for Apache Airflow (Amazon MWAA) Amazon MWAA is a managed service for Apache Airflow, which is an open-source workflow automation tool. Apache Airflow can be used both on-premises and in the cloud, making it ideal for hybrid environments. Using Amazon MWAA allows the data engineer to leverage the managed service in the cloud while maintaining the ability to use the same open-source Airflow setup on-premises, ensuring portability and consistency across environments.
AMWAA is just Apache Airflow managed by AWS. Using it means you can use the same dags for your on-premises solution that you also use on cloud