Sending 1 Million Emails in 25 Minutes
High-Performance Mautic on AWS
About the Project
This project is part of a larger multi-tenant SaaS platform where each tenant runs a white-labeled Mautic instance. As the platform grew, the default Mautic email pipeline — which sends emails one by one via SMTP — became a critical bottleneck. Campaigns with hundreds of thousands of recipients took hours to complete or simply timed out. We needed to rebuild the email sending pipeline from scratch to support production-scale volumes.
Performance Results
The Challenge
Building a high-performance email pipeline for a multi-tenant Mautic platform required solving several interconnected problems simultaneously:
Send 1M+ emails in under 30 minutes
Dynamic content personalization per recipient
Maintain deliverability at high volume
Handle AWS SQS 256KB message size limit
Process bounces across multiple tenants
Cost-efficient execution on spot instances
The Email Pipeline
We replaced Mautic's default one-by-one SMTP sending with a three-stage parallel pipeline designed for throughput.
Segment Scanning
Mautic scans the campaign segment and batches millions of contacts into queue messages. This stage completes in milliseconds to seconds regardless of segment size, thanks to optimized database queries that only extract contact IDs.
Payload Assembly
52 parallel worker pods consume messages from the queue. Each worker assembles the full email payload: resolving dynamic content, personalizing templates per recipient, and batching up to 1,000 recipients per Mailgun API call.
Mailgun Batch API
Assembled batches are dispatched to Mailgun's batch API endpoint. Each API call handles up to 1,000 recipients with per-recipient variable substitution, eliminating the need for individual SMTP connections.
Engineering Challenges
Custom SQS + S3 Transport
AWS SQS has a 256KB message size limit. Email payloads with personalized HTML content easily exceed this. We built a custom transport layer: the full payload is stored in S3, and the SQS message contains only a reference to the S3 object. Workers fetch the payload from S3 on consumption. This eliminates the size constraint while maintaining queue reliability.
Intelligent Bounce Handling
With hundreds of tenants sharing a Mailgun domain, bounce notifications needed routing to the correct tenant instance. We implemented a custom header injection system: each outgoing email carries a tenant identifier header. A polling-based bounce processor reads Mailgun's event stream, extracts the tenant ID, and routes the bounce to the correct Mautic instance for contact status updates.
Graceful Processing Under Spot Interruptions
Worker pods run on spot instances for cost savings, but AWS can reclaim these with 2 minutes notice. We implemented graceful shutdown handling: when a termination signal arrives, workers stop consuming new messages, finish processing their current batch, and write checkpoints. A force-kill fallback ensures no orphaned processes. Messages from interrupted workers return to the queue automatically.
Email Deduplication & Queue Priority
With 52 parallel workers and thousands of queue messages, preventing duplicate sends is critical. We implemented a Redis-based deduplication layer that tracks processed contact IDs across all workers in real-time. Queue priority ensures time-sensitive campaigns are processed before bulk sends.
Business Impact
Lower infrastructure costs than initially projected thanks to spot instance optimization
Deliverability protected through proper API-based sending and reputation management
Zero email loss even during spot instance interruptions
Custom SQS + S3 transport pattern reusable across the platform
Architecture scalable to support 300-500 tenants sending simultaneously
Tech Stack
Need High-Performance Email at Scale?
Let's talk about optimizing your Mautic email pipeline for production-scale volumes.
Talk to an Expert