AWS Certified Solutions Architect - Associate SAA-C03 Exam - Question 98

Question

An image-processing company has a web application that users use to upload images. The application uploads the images into an Amazon S3 bucket. The company has set up S3 event notifications to publish the object creation events to an Amazon Simple Queue Service (Amazon SQS) standard queue. The SQS queue serves as the event source for an AWS Lambda function that processes the images and sends the results to users through email.

Users report that they are receiving multiple email messages for every uploaded image. A solutions architect determines that SQS messages are invoking the Lambda function more than once, resulting in multiple email messages.

What should the solutions architect do to resolve this issue with the LEAST operational overhead?

Examice · Accepted Answer

The users are receiving multiple email messages because the Lambda function is being invoked more than once for the same SQS message. This occurs when the visibility timeout of the SQS queue is too short. If the visibility timeout ends before the Lambda function completes processing the message, the message will become visible again in the queue, leading to its reprocessing and the sending of multiple emails. Increasing the visibility timeout in the SQS queue to a value greater than the total of the function timeout and the batch window timeout ensures that each message is given sufficient time for processing before it can be seen again by another consumer. Therefore, the issue can be resolved by increasing the visibility timeout.

Six_Fingered_Jose · Answer

answer should be C,
users get duplicated messages because -> lambda polls the message, and starts processing the message.
However, before the first lambda can finish processing the message, the visibility timeout runs out on SQS, and SQS returns the message to the poll, causing another Lambda node to process that same message.
By increasing the visibility timeout, it should prevent SQS from returning a message back to the poll before Lambda can finish processing the message

brushek · Answer

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html

this is important part:
Immediately after a message is received, it remains in the queue. To prevent other consumers from processing the message again, Amazon SQS sets a visibility timeout, a period of time during which Amazon SQS prevents other consumers from receiving and processing the message. The default visibility timeout for a message is 30 seconds. The minimum is 0 seconds. The maximum is 12 hours.

MrAWS · Answer

Increasing the visibility timeout only stops other consumers of the queue from seeing that message until it is handled and deleted.

However in our case

- there are duplicate messages in the queue!! So I do not see how increasing the visibility handles this issue. 
- The question clearly calls out that a 'standard queue' is being used  so the reader will think of this issues caused by a standard queue... which is order of order and DEDUPLICATION.

The also do not mention performance as an issue, which might be a reason not to use FIFO.

The only issue I have with 'B' as an answer is it says

'change' the standard to FIFO.  technically you cannot switch to a FIFO queue once its created... but you can at a higher level change the architecture to use a FIFO queue.

MXB05 · Answer

A can not be correct, long polling will only ensure that all images are retrieved from all SQS servers in one query. If the same message triggers the lambda function twice it is likely because the visibility timeout isn't long enough and lambda didn't repsond in time with a deletion of the message ->> C is correct

PaoloRoma · Answer

The only options that can rule out duplicated messages is B) as per doc "Unlike standard queues, FIFO queues don't introduce duplicate messages. FIFO queues help you avoid sending duplicates to a queue."
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues-exactly-once-processing.html

Answer C, though with less ops overheads, doesn't guarantee to rule out the event to send multiple emails related to the same image. This will avoid (minimise) processing the same message two or more times, however do not solve the problem of duplicated messages.

Olaunfazed · Answer

Answer is B.

B. Change the SQS standard queue to an SQS FIFO queue. Use the message deduplication ID to discard duplicate messages.

By changing the SQS standard queue to an SQS FIFO (First-In-First-Out) queue, you can ensure that messages are processed in the order they are received and that each message is processed only once. FIFO queues provide exactly-once processing and eliminate duplicates.

Using the message deduplication ID feature of SQS FIFO queues, you can assign a unique identifier (such as the S3 object key) to each message. SQS will check the deduplication ID of incoming messages and discard duplicate messages with the same deduplication ID. This ensures that only unique messages are processed by the Lambda function.

This solution requires minimal operational overhead as it mainly involves changing the queue type and using the deduplication ID feature, without requiring modifications to the Lambda function or adjusting timeouts.

Nandan747 · Answer

At first I thought the answer should be B, since they specifically mentioned it is a Standard Queue and we know that in Std queue, we do get some duplicates. But the real catch over here is EVERY time the users are getting duplicate. So it must be the VisibilityTimeout issue  which isn't long enough so EVERY time the message goes back on the queue before processing by one Lambda is completed and at the same time is being picked up by another function for processing.

Abrar2022 · Answer

FIFO - IS A SOLUTION BUT REQUIRES OPERATIONAL OVERHEAD.
INCREASING VISIBILITY TIMEOUT -  REQUIRES FAR LESS OPERATIONAL OVERHEAD.

Ruffyit · Answer

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html

this is important part:
Immediately after a message is received, it remains in the queue. To prevent other consumers from processing the message again, Amazon SQS sets a visibility timeout, a period of time during which Amazon SQS prevents other consumers from receiving and processing the message. The default visibility timeout for a message is 30 seconds. The minimum is 0 seconds. The maximum is 12 hours

ManikRoy · Answer

to all those who are considering option B, Its INCORRECT.
Amazon Simple Queue Service FIFO (First-In-First-Out) queues aren't supported as an Amazon S3 event notification destination.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/EventNotifications.html

rob74 · Answer

I  exlude Polling because-->"The maximum long polling wait time is 20 seconds"
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-short-and-long-polling.html

Buruguduystunstugudunstuy · Answer

Option D, The solution architect should modify the Lambda function to delete each message from the SQS queue immediately after the message is read before processing. This is the least operationally overhead solution because it does not require any changes to the SQS queue or any additional configuration.

naabe · Answer

LEAST operational overhead Option D

lovelazur · Answer

To address the issue of users receiving multiple email messages for every uploaded image with the least operational overhead, increasing the visibility timeout in the SQS queue is the best solution. This requires no additional configuration and thus has the least operational overhead compared to other options. However, this solution does not completely prevent duplicates, so there is still a possibility of duplicate emails being sent. While using a FIFO queue can prevent duplicates, it requires additional configuration and therefore may have higher operational overhead.

kuls91 · Answer

I took the exam in April 2023, out of 65 Questions only 25-30 were from the dumps. I got passed (820!) but these dumps are singly not reliable. I thing is sure, going through these questions and working itself for the answers will help to pass the actual exam.

quanbui · Answer

ccccccc

Bmarodi · Answer

I go for option C.

cookieMr · Answer

A. Long polling doesn't directly address the issue of multiple invocations of the Lambda for the same message. Increasing the ReceiveMessage may not completely prevent duplicate invocations.

B. Changing the queue type from standard to FIFO requires additional considerations and changes to the application architecture. It may involve modifying the event configuration and handling message deduplication IDs, which can introduce operational overhead.

D. Deleting messages immediately after reading them may lead to message loss if the Lambda encounters an error or fails to process the image successfully. It does not guarantee message processing and can result in data loss.

C. By setting the visibility timeout to a value greater than the total time required for the Lambda to process the image and send the email, you ensure that the message is not made visible to other consumers during processing. This prevents duplicate invocations of the Lambda for the same message.

prabhjot · Answer

ans B - Option A (long polling), Option C (increasing visibility timeout), and Option D (deleting messages immediately) do not address the root cause of the problem, which is the duplication of messages in the queue.

0xE8D4A51000 · Answer

All answers are wrong, the answer B

shwelin · Answer

I will stick with "B", the correct answer.

824c449 · Answer

B: Directly addresses the issue of duplicates by ensuring exact-once processing and message ordering, which mitigates the risk without requiring adjustments based on anticipated processing times or additional application logic to manage potential duplicates.

So, while increasing the visibility timeout could help in managing when messages are available for processing again, it doesn't provide a structural solution to the problem of duplicate processing in the way that using an SQS FIFO queue does, nor does it ensure the operational simplicity and reliability that comes with eliminating duplicates at the source.

Solomon2001 · Answer

B. Change the SQS standard queue to an SQS FIFO queue. Use the message deduplication ID to discard duplicate messages.

Here’s why this option is suitable:

SQS FIFO Queue:
Amazon SQS FIFO (First-In-First-Out) queues ensure that the order of messages is preserved.
Each message in a FIFO queue has a unique message deduplication ID.
By using a FIFO queue, you can prevent duplicate messages from being processed.
Message Deduplication ID:
When sending messages to a FIFO queue, set the message deduplication ID to ensure that identical messages are treated as duplicates.
If a message with the same deduplication ID is sent within a 5-minute window, it is considered a duplicate and discarded.
https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html

PaulGa · Answer

Ans D - is my preferred choice because it removes the record and therefore eliminates the possibility of duplication; the only snag here is ensuring the SQS record is locked.  
Ans C, which has the most votes will work but is less robust - but perhaps it is the cheapest option.  
Ans A, the author's preference, won't work unless the Lambda function is guaranteed to complete within 30secs - and nowhere is that stated

AwsAbhiKumar · Answer

I believe answer is B cause FIFO queues provide exactly-once processing and guarantee that messages are delivered in order. They also support message deduplication, which prevents processing duplicate messages within a specified deduplication interval.

Whereas a longer visibility timeout might prevent messages from being processed more than once if processing takes longer than expected. However, it does not eliminate the inherent at-least-once delivery behavior of standard queues, which can still cause duplicates.

Hunkie · Answer

https://www.examtopics.com/discussions/amazon/view/83096-exam-aws-certified-solutions-architect-associate-saa-c02/

Wpcorgan · Answer

C is correct

career360guru · Answer

Option C is the most probable case.
Though option B can also cause some duplicates but given this is happening for every request/users C seems to be real root cuase.

Richaqua · Answer

Since SQS queue does not delete the message by default, Lambda function can be modified to delete the messages after it has been processed.

psr83 · Answer

https://aws.amazon.com/sqs/faqs/
SQS - LongPolling decreases the number of API calls made to SQS while increasing the efficiency and reducing latency of your application 
Long polling reduces the number of empty responses by allowing Amazon SQS to wait a specified time for a message to become available in the queue before sending a response. Also, long polling eliminates false empty responses by querying all of the servers instead of a sampling of server

PassNow1234 · Answer

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html

still valid

techhb · Answer

C is right answer here
https://www.examtopics.com/discussions/amazon/view/83096-exam-aws-certified-solutions-architect-associate-saa-c02/

MrAWS · Answer

the only thing that addresses deduplication is using a FIFO queue OR by coding idempotency into your code. Increasing the visibility timeout only means you can delete the message you were processing, it doesn't handle the duplicates and therefore doesn't answer the question of

"What should the solutions architect do to resolve this issue "

Steve_4542636 · Answer

Here https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html it says this for sqs standard.  For standard queues, the visibility timeout isn't a guarantee against receiving a message twice. For more information, see At-least-once delivery.

osmk · Answer

C In an application under heavy load or with spiky traffic patterns, it’s recommended that you:

Set the queue’s visibility timeout to at least six times the function timeout value. This allows the function time to process each batch of records if the function execution is throttled while processing a previous batch.https://docs.aws.amazon.com/lambda/latest/operatorguide/sqs-retries.html

kraken21 · Answer

Key is minimal operational overhead.

tikytaka · Answer

C is wrong:
'When the wait time for the ReceiveMessage API action is greater than 0, long polling is in effect. The maximum long polling wait time is 20 seconds.'

tikytaka · Answer

Apologies, I meant A is wrong

Rahulbit34 · Answer

SQS VISIBILITY TIMEOUT can help preventing the reprocessing of the message from the queue. By default the timeout is 30 secs, min 0 and max is 12 hours.

Guru4Cloud · Answer

I would go with the C.

TariqKipkemei · Answer

Immediately after a message is received, it remains in the queue. To prevent other consumers from processing the message again, Amazon SQS sets a visibility timeout, a period of time during which Amazon SQS prevents all consumers from receiving and processing the message. The default visibility timeout for a message is 30 seconds. The minimum is 0 seconds. The maximum is 12 hours.

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html#:~:text=SQS%20sets%20a-,visibility%20timeout,-%2C%20a%20period%20of

kwang312 · Answer

D is an incorrect answer because Lamda automatically deletes message from the queue when finish process

hieulam · Answer

I think A is correct.
https://aws.amazon.com/blogs/developer/polling-messages-from-a-amazon-sqs-queue/#:~:text=When%20disabling,more%20API%20calls.

vijaykamal · Answer

Long polling is incorrect...it just means that SQS queue is connected after specific interval instead of looking for messages in queue in very short interval...long polling saves money but does not help to remove duplicate.

Correct Answer: C

Anonymous · Answer

least operational overheads

awsgeek75 · Answer

In this setup the only way to get multiple emails is when same image is processed multiple times. This only happens when another Lambda starts processing while the previous one hasn't finished processing. 
Increasing the SQS queue timeout to be greater than Lambda timeout will ensure that other Lambda can't see the SQS message before previous Lambda finishes processing or times out.
So C is best answer
A: Long polling won't fix anything
B: FIFO is nice idea but how will the Lambda function know it has got a duplicate message?
D: Wrong as in case of Lambda timeout that message is lost without being processed

vip2 · Answer

'Visibility Timeout' is suitable(better) solution to solve the issue.

Uzbekistan · Answer

Given the requirement to resolve the issue of multiple email messages being sent to users with the least operational overhead, the most appropriate solution is:

B. Change the SQS standard queue to an SQS FIFO queue. Use the message deduplication ID to discard duplicate messages.

Explanation:

SQS FIFO Queue: FIFO (First-In-First-Out) queues in SQS ensure that the order in which messages are sent and received is strictly preserved and that each message is processed only once. By switching to an SQS FIFO queue, you can prevent the Lambda function from processing duplicate messages.

jaradat02 · Answer

the correct answer is C, although B and D might also work, C is the one that achieves the least operational overhead condition.

JA2018 · Answer

it should be B

-	Using a FIFO (First In, First Out) queue ensures that messages are processed in the exact order they are received, preventing duplicates from being processed multiple times by the Lambda function. By setting up message deduplication IDs, you can further guarantee that even if the same message is sent multiple times, only one instance will be processed.

-	When dealing with potential duplicate messages in an event-driven architecture, switching to an SQS FIFO queue with message deduplication is generally the most efficient and reliable solution to prevent redundant processing.

salman7540 · Answer

FIFO queues are designed to never introduce duplicate messages. However, a message producer might introduce duplicates if another producer (lambda in this case) picks up same fifo message after visibility timeout over but previous lambda was still processing it. Hence correct answer is C.

satyaammm · Answer

Using Visibility Timeout is the most suitable option here with the least operational overhead.

Rcosmos · Answer

C. Aumentar o tempo limite de visibilidade:

Um tempo limite de visibilidade maior reduz a chance de reprocessamento prematuro, mas não resolve completamente o problema de mensagens duplicadas.
É mais complexo ajustar adequadamente o tempo de visibilidade para sincronizar com o processamento da função Lambda e o tempo limite.

zdi561 · Answer

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html , standard queue is delivered at lease once, the duplication can not be avoided

Dharmarajan · Answer

SQS feature of "Visibility timeout" sets the time for getting acknowledgement from processor that the message has been successfully processed. I think it should have been called "Invisibility " rather than "Visibility" timeout. Anyways, I think AWS engineers have thought this Queue service through well.  So if the processing time> visibility timeout, the message comes back and is picked up by another execution of lambda funtion. It is such a small thing but it can easily rake up lambda usage if not considered!

AWS Certified Solutions Architect - Associate SAA-C03 Exam - Question 98

Discussion