A company stores its application logs in an Amazon CloudWatch Logs log group. A new policy requires the company to store all application logs in Amazon OpenSearch Service (Amazon Elasticsearch Service) in near-real time.
Which solution will meet this requirement with the LEAST operational overhead?
Configuring a CloudWatch Logs subscription to stream the logs to Amazon OpenSearch Service provides a direct and efficient way to meet the requirement with minimal operational overhead. This solution enables near-real-time streaming of logs without the need for additional services or complex configurations, ensuring simplicity and reliability in the log transfer process.
answer is A https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_OpenSearch_Stream.html > You can configure a CloudWatch Logs log group to stream data it receives to your Amazon OpenSearch Service cluster in NEAR REAL-TIME through a CloudWatch Logs subscription least overhead compared to kinesis
Great link. Convinced me
Option A (Configure a CloudWatch Logs subscription to stream the logs to Amazon OpenSearch Service (Amazon Elasticsearch Service)) is not a suitable option, as a CloudWatch Logs subscription is designed to send log events to a destination such as an Amazon Simple Notification Service (Amazon SNS) topic or an AWS Lambda function. It is not designed to write logs directly to Amazon Elasticsearch Service (Amazon ES).
that is not true, you can stream logs from CloudWatch Logs directly to OpenSearch
Zerotn3 is right! There should be a Lambda for writing into ES
good enough for me
The correct answer is C: Create an Amazon Kinesis Data Firehose delivery stream. Configure the log group as the delivery stream source. Configure Amazon OpenSearch Service (Amazon Elasticsearch Service) as the delivery stream's destination. This solution uses Amazon Kinesis Data Firehose, which is a fully managed service for streaming data to Amazon OpenSearch Service (Amazon Elasticsearch Service) and other destinations. You can configure the log group as the source of the delivery stream and Amazon OpenSearch Service as the destination. This solution requires minimal operational overhead, as Kinesis Data Firehose automatically scales and handles data delivery, transformation, and indexing.
Option A: Configure a CloudWatch Logs subscription to stream the logs to Amazon OpenSearch Service (Amazon Elasticsearch Service) would also work, but it may require more operational overhead as you would need to set up and manage the subscription and ensure that the logs are delivered in near-real time. Option B: Create an AWS Lambda function. Use the log group to invoke the function to write the logs to Amazon OpenSearch Service (Amazon Elasticsearch Service) would also work, but it may require more operational overhead as you would need to set up and manage the Lambda function and ensure that it scales to handle the incoming logs. Option D: Install and configure Amazon Kinesis Agent on each application server to deliver the logs to Amazon Kinesis Data Streams. Configure Kinesis Data Streams to deliver the logs to Amazon OpenSearch Service (Amazon Elasticsearch Service) would also work, but it may require more operational overhead as you would need to install and configure the Kinesis Agent on each application server and set up and manage the Kinesis Data Streams.
https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_OpenSearch_Stream.html
ANSWER A https://docs.aws.amazon.com/opensearch-service/latest/developerguide/integrations.html You can use CloudWatch or Kinesis, but in the Kinesis description it never says real time, however in the Cloudwatch description it does say Real time ""You can load streaming data from CloudWatch Logs to your OpenSearch Service domain by using a CloudWatch Logs subscription . For information about Amazon CloudWatch subscriptions, see Real-time processing of log data with subscriptions.""
Since the scenario perfectly fits this description: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_OpenSearch_Stream.html
Correct Answer: C. Create an Amazon Kinesis Data Firehose delivery stream. Configure the log group as the delivery stream's source. Configure Amazon OpenSearch Service (Amazon Elasticsearch Service) as the delivery stream's destination. Explanation: Amazon Kinesis Data Firehose is a fully managed service for delivering real-time streaming data to destinations such as Amazon OpenSearch Service. It requires minimal setup and management, making it a low-overhead solution. By configuring the log group as the source for the Kinesis Data Firehose delivery stream and Amazon OpenSearch Service as the destination, logs can be delivered in near-real time with built-in reliability and scalability.
easy enough to figure out. Option A
A for sure
It can natively connect to CloudWatch Logs as a source and OpenSearch Service as a destination, handling the delivery of logs efficiently and with minimal setup. This approach offers the least operational overhead by simplifying the data transfer pipeline with automatic scaling and error handling.
You can configure a CloudWatch Logs log group to stream data it receives to your Amazon OpenSearch Service cluster in near real-time through a CloudWatch Logs subscription. here is the link/; https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_OpenSearch_Stream.html
Answer A. This doc clarifies the subject: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_OpenSearch_Stream.html
A is correct You can configure a CloudWatch Logs log group to stream data it receives to your Amazon OpenSearch Service cluster in near real-time through a CloudWatch Logs subscription. https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_OpenSearch_Stream.html
You can configure a CloudWatch Logs log group to stream data it receives to your Amazon OpenSearch Service cluster in near real-time through a CloudWatch Logs subscription. This is the solution that requires the least operational overhead. https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_OpenSearch_Stream.html
A, You can configure a CloudWatch Logs log group to stream data it receives to your Amazon OpenSearch Service cluster in near real-time through a CloudWatch Logs subscription
Amazon Kinesis Data Firehose can automatically deliver logs from CloudWatch Logs to Amazon OpenSearch Service without requiring you to manage and configure additional components or write custom code. It simplifies the process and reduces operational overhead
Following these key words: - near-real time. - LEAST operational overhead and the fact that CloudWatch loggroup support OpenSearch Service subscription filter
Arratum: Obviously option A according to my wording
The solution that will meet the requirement with the least operational overhead is: **Option A**: Configure a CloudWatch Logs subscription to stream the logs to Amazon OpenSearch Service (Amazon Elasticsearch Service). This option allows you to directly stream logs from CloudWatch to OpenSearch Service (Elasticsearch Service) in near-real time without the need for additional services or resources, thus minimizing operational overhead. The other options involve additional services (Lambda, Kinesis Data Firehose, Kinesis Data Streams) and would therefore require more operational management.
A, C can both support near-real-time logs transfer to OpenSearch. But it depends on the current needs. Based on the context of question, Option A is the best one. For Option C: This Kinesis Data Firehose offers additional benefits like easy scaling, built-in failure handling, and potential for data transformation if needed. But these are not required by the question. It only requires LEAST overhead-operation and near-real-time transfer then A is straightforward.
You need real time buffer like Kinesis, otherwise you are going to lose data.
Pretty sure A supports near-real-time transfer
They both do, but C has the least operational overhead.