A solutions architect is designing a solution to process events. The solution must have the ability to scale in and out based on the number of events that the solution receives. If a processing error occurs, the event must move into a separate queue for review.
Which solution will meet these requirements?
The best solution for processing events with automatic scaling based on the number of events and the ability to handle processing errors by moving events to a separate queue is to use Amazon SNS with an AWS Lambda function. In this configuration, events are sent to an Amazon SNS topic, which triggers an AWS Lambda function to process the events. AWS Lambda automatically scales based on the number of events, ensuring efficient handling of varying event loads. Additionally, AWS Lambda supports the configuration of an on-failure destination, where a separate Amazon SQS queue can be set as the target for events that fail to process. This setup meets all the requirements: it scales efficiently based on the number of events and provides a separate queue for reviewing processing errors.
I would go with B just because of the wording. I believe A should work just fine, but the question asks for "scale in and out based on the number of events." In my opinion, that is what SNS->Lambda->SQS(DLQ) would do, too; I think the SNS->Lambda scale in/out behavior is more implicit. So I will go with B here because it is more explicit.
Configuring scaling based on the age of the oldest message is nowhere near as good as scaling based on size of the Queue for this use case. age of the oldest message will grow linearly based on time. If there is a dramatic spike in the Queue size due to increased traffic, like 100X increase in size. Then the queue will have grown a lot but the oldest message will only increase in age linearly, so the scaling will not be able to realize how much the workload has increased.
makes sense
very good explanation. Moreover, go serverless as much as possible. EC2 vs Lambda - Lamda is always preferred.
ApproximateAgeOfOldestMessage metric may not be as responsive as needed, and it doesn't directly address the requirement for handling processing errors by moving events to a separate queue for review.
I would go with A. Scaling in/out based on message age does not align with what question asks i.e. it should be based on the number of events. So, B is not right here.
Scaling based on how long someone is waiting is another way of basing it on the number of events, but I see what you mean. Lambda will scale based on the number 1:1 and B will scale in whatever configuration you want based on time, not number of events specifically.
I would go for A as the question mentions specifically about scaling based on the 'number of events' and option B goes for age of theoldest message in the queue. Option B does sound deliberate to distract.
Actually I would go with B based on this https://repost.aws/knowledge-center/sqs-troubleshoot-oldestmessage-metric#:~:text=If%20the%20queue%20has,to%20maintain%20the%20backlog.
A fulfils all objectives. In B, ApproximateAgeOfOldestMessage doesn't translate to a reliable scaling pattern, and EC2s are implied. C does not implement a dead-letter queue D is overengineered.
I prefer A as the solution is serverless.
Option A is better with assumption that SNS will scale in and scale out Lambda depending on number of incoming messages.
Option B is the right answer.
Vote for A
The question typical AWS thrown words and leaving gaps.. but still going for B
Answer is B I would go with A, except a Dead Letter Q is not an SQS queue. There are only two types of SQS Queues, namely, Standard and FIFO. A DLQ is a special message queue (not SQS). See here for confirmation: https://aws.amazon.com/what-is/dead-letter-queue/#:~:text=A%20dead%2Dletter%20queue%20(DLQ)%20is%20a%20special%20type,communication%20in%20a%20distributed%20system.
I do think the B is correct way with ApproximateNumberOfMessages instead of ApproximateAgeOfOldestMessage. there is no such metric ...AgeOfOldestMessage
publishing events to an SQS queue, creating an EC2 Auto Scaling group that scales based on the queue's ApproximateAgeOfOldestMessage metric, and configuring the application to write failed messages to a dead-letter queue provides a scalable, fault-tolerant, and cost-effective solution for event processing with the ability to handle processing errors separately.
A seems better. Dont really know.
Option B. Question states, " must move to into a separate queue for review" Dead-Letter queues give you this capability for debugging or troubleshooting the issue. https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html
SQS with DLQ
B is the answer --- SQS is the right tool for the job.