AWS SQS (Simple Queue Service) - Interview Guide
Amazon SQS is a fully managed message queuing service that enables decoupling and scaling of microservices, distributed systems, and serverless applications. SQS offers two queue types: Standard (high throughput, at-least-once delivery) and FIFO (ordered per message group, with send-side deduplication). AWS calls FIFO “exactly-once processing”, which means duplicate sends within a 5-minute window are dropped; consumers can still see a message twice after a crash, so they stay idempotent.
Key Features
- Fully managed: No server management
- Scalable: Handles any volume of messages
- Standard & FIFO Queues: Choose based on ordering and deduplication needs
- Dead Letter Queues (DLQ): Handle message failures
- Visibility Timeout: Prevents multiple consumers from processing the same message
- Long Polling: Reduces empty responses and cost
- Server-side Encryption: Protects sensitive data
Architecture
Producer → SQS Queue → Consumer(s)
| | |
API Standard/FIFO Lambda, EC2, ECS, etc.- Producer: Sends messages to the queue
- Queue: Stores messages until processed
- Consumer: Retrieves and processes messages
Use Cases
- Decoupling microservices
- Task scheduling and background jobs
- Buffering and load leveling
- Order processing
- Event-driven serverless workflows
Basic Usage Example
import boto3
sqs = boto3.client('sqs')
queue_url = 'https://sqs.us-east-1.amazonaws.com/123456789012/my-queue'
# Send a message
sqs.send_message(QueueUrl=queue_url, MessageBody='Hello, SQS!')
# Receive messages
response = sqs.receive_message(QueueUrl=queue_url, MaxNumberOfMessages=1)
for msg in response.get('Messages', []):
print(msg['Body'])
# Delete after processing
sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=msg['ReceiptHandle'])Best Practices
- Use DLQs for error handling
- Set appropriate visibility timeout
- Use batching for high throughput
- Monitor queue length and age
- Secure queues with IAM policies
- Use FIFO queues for strict ordering and deduplication
Security
- IAM Policies: Restrict access to queues
- SSE: Enable server-side encryption for sensitive data
- VPC Endpoints: Private connectivity
Monitoring
- CloudWatch Metrics: Monitor queue length, age, and failed messages
- CloudTrail: Audit API calls
Cost Optimization
- Use long polling to reduce API calls
- Delete processed messages promptly
- Clean up unused queues
Common Interview Questions
Q: SQS vs SNS?
- SQS: Queue, pull-based, one-to-one
- SNS: Pub/sub, push-based, one-to-many
Q: How does SQS guarantee delivery?
- Standard: At-least-once, possible duplicates
- FIFO: ordered per group; send-side dedup within 5 minutes (consumers still idempotent)
Q: What is a Dead Letter Queue?
- A secondary queue for messages that can’t be processed after max attempts
Q: How do you handle message ordering?
- Use FIFO queues with MessageGroupId
Q: How do you avoid message loss?
- Use DLQs, monitor metrics, set proper visibility timeout
Interview angle 5
- “What is the visibility timeout and how do you get it wrong?” - the window during which a received message is hidden from other consumers. Set it shorter than your processing time and a second consumer picks up the same message, so the work runs twice. Set it far too long and a crashed consumer’s messages stall. Extend it with a heartbeat for variable-length work.
- “Standard or FIFO?” - standard gives near-unlimited throughput, at-least-once delivery and best-effort ordering. FIFO gives send-side deduplication within a 5-minute window and strict order per message group (a consumer crash before
DeleteMessagestill redelivers), at a much lower throughput ceiling. Most systems take standard plus idempotent consumers. - “How do you handle a poison message?” - a redrive policy to a dead-letter queue after N receives, then alarm on DLQ depth. Without one, a message that always fails is retried forever and blocks throughput.
- “Long or short polling?” - long polling (
WaitTimeSecondsup to 20). Short polling samples a subset of servers, so it returns empty responses even when messages exist, and it costs more in API calls. - “How do you scale consumers?” - on queue depth or age of the oldest message, not CPU. Age of oldest message is the better signal because it reflects user-visible lag.