horizontal scaling agent inboxes: how to grow from one inbox to thousands

You started with one agent and one inbox. It worked fine. Then you added five agents, then twenty, and now you're staring at a spreadsheet of inbox assignments wondering why half your agents are stepping on each other's replies.

This is the moment most teams discover that their inbox architecture doesn't scale sideways. It scales up (bigger server, more memory, faster polling) until it doesn't. Horizontal scaling for agent inboxes is a different approach: instead of making one thing bigger, you spread the work across many independent units. For AI agents that self-provision their own email, this distinction matters more than it does for traditional web apps.

If you'd rather skip the manual setup, LobsterMail handles this in about a minute.

Horizontal scaling vs vertical scaling for agent inboxes#

The core difference is direction. Vertical scaling means giving a single agent (or a single inbox) more resources. Horizontal scaling means adding more agents and more inboxes, distributing the load across them.

Here's how the two approaches compare when applied specifically to agent inbox infrastructure:

If your agent handles fewer than a hundred emails per day, vertical scaling is probably fine. You don't need to architect for problems you don't have. But once you're running 50 agent inboxes or more, horizontal scaling becomes less of an optimization and more of a requirement.

Why agent inboxes are different from traditional email servers#

Traditional email scaling is about storage and throughput: how many mailboxes can the server hold, how fast can it process inbound SMTP connections. The server itself is passive. It receives mail, stores it, and waits for a client to fetch it.

Agent inboxes flip this model. The inbox isn't just storage. It's the trigger point for autonomous behavior. When an email arrives, the agent reads it, decides what to do, and acts. That means scaling agent inboxes isn't just about handling more mail. It's about handling more decisions per second.

This creates two problems that traditional email infrastructure never had to solve:

Problem one: duplicate processing. If two agent instances poll the same inbox, they might both grab the same email and both reply to it. The customer gets two identical responses. Or worse, two contradictory ones. Traditional IMAP handles this with flags and locks, but agents operating at speed need something more deliberate.

Problem two: conversation continuity. Email threads depend on In-Reply-To and References headers. If instance A handles the first message in a thread and instance B handles the follow-up, instance B has no context. It doesn't know what instance A said. The reply might contradict the previous message or repeat information the customer already received.

Stateless vs stateful inbox design#

Your ability to scale horizontally depends almost entirely on whether your agents are stateless or stateful.

A stateful agent keeps conversation context in local memory. It remembers what it said last, what the customer asked, and what step of the workflow it's on. This works beautifully for a single instance. It falls apart the moment you add a second one, because the second instance doesn't share that memory.

A stateless agent stores all context externally: in a database, a shared cache, or embedded in the email thread itself. Any instance can pick up any message and reconstruct the full conversation from the external store. This is harder to build initially, but it's the only architecture that scales horizontally without coordination nightmares.

If you're building agents that need to scale, design them stateless from day one. Retrofitting statefulness out of an agent that assumes local memory is painful and error-prone.

Inbox sharding strategies#

Once your agents are stateless, you need a strategy for distributing inboxes across instances. There are three common patterns:

One inbox per instance. The simplest model. Each agent instance creates and owns its own inbox. No sharing, no contention, no duplicate processing. The downside is that you need a routing layer to direct inbound email to the right inbox. For agents that create their own inboxes on signup, this happens naturally: the agent provisions a fresh address and hands it to whatever service it's interacting with.

Shared inbox with claim-based processing. Multiple instances watch a single inbox, but each message is "claimed" by one instance before processing. This works like a job queue. The upside is simpler routing (one address for everything). The downside is that you need a reliable claim mechanism to prevent duplicates. A message broker like Redis or SQS sitting between the inbox and the agents handles this well.

Topic-based sharding. Inbound mail is routed to different inboxes based on content, sender, or subject. Support emails go to one pool of agents, verification codes go to another, notifications go to a third. Each pool scales independently. This is the most operationally complex approach, but it lets you allocate resources where they're actually needed instead of scaling everything uniformly.

For most teams, starting with one inbox per instance and moving to topic-based sharding as volume grows is the right progression. Don't over-architect early.

How to handle reply threading across instances#

The In-Reply-To header is what keeps email threads coherent. When your agent replies to a message, the reply needs to reference the Message-ID of the original. If a different instance handles the next inbound message in that thread, it needs to know what was already said.

Two approaches work:

Store outbound messages externally. Every time an agent instance sends a reply, it logs the full message (including headers) to a shared database. When any instance receives a follow-up, it queries the database for the thread history before responding. This adds a database lookup per inbound message, but it guarantees continuity.

Use the inbox as the source of truth. Instead of a separate database, have the agent re-read its own inbox to reconstruct the thread. LobsterMail's receive() method returns the full thread if messages are in the same conversation. This is simpler but slower for long threads.

Scaling triggers and observability#

You need to know when to add capacity. The three metrics that matter most for horizontal scaling of agent inboxes:

Inbox queue depth. How many unread messages are sitting in each inbox? If this number trends upward over time, your agents can't keep up with inbound volume. Add instances.

Response latency. How long between an email arriving and the agent sending a reply? If this exceeds your SLA (or your customers' patience), you need more processing capacity.

Error rate on sends. A spike in 4xx or 5xx errors when sending outbound email often means you're hitting rate limits. This is where horizontal scaling helps directly: spreading sends across more inboxes means each inbox stays within its rate limits. If you're choosing between webhooks and polling for inbound delivery, webhooks give you better real-time observability since you can measure delivery-to-processing latency without polling intervals muddying the numbers.

Set up alerts on all three. Autoscaling based on queue depth is the most reliable trigger, because it directly measures whether your agents are keeping up.

How LobsterMail's architecture supports horizontal scaling#

LobsterMail was built for agents that provision their own infrastructure. That design decision has a side effect: it makes horizontal scaling straightforward.

When an agent calls LobsterMail.create() followed by createSmartInbox(), it gets a dedicated inbox with no shared state and no coordination with other instances. Spin up ten instances, and each one creates its own inbox. No configuration, no DNS changes, no shared credentials. The free tier gives you an inbox with 1,000 emails per month. The Builder tier at $9/month gets you up to 10 inboxes with higher send limits.

Because inboxes are created programmatically, your orchestration layer (Kubernetes, ECS, a simple process manager) can treat inbox provisioning as part of the startup sequence. Instance boots, creates inbox, starts processing. Instance shuts down, inbox goes idle. No human touches anything.

This is the difference between email infrastructure that was designed for humans logging into webmail and infrastructure designed for agents that scale on their own. The agent doesn't need to ask for permission or wait for provisioning. It hatches its own inbox and starts working.

When vertical scaling is the better choice#

Not every workload needs horizontal scaling. If your agent does heavy per-message processing (running an LLM on every inbound email, performing multi-step reasoning, calling external APIs), the bottleneck might be compute per message rather than messages per second. In that case, giving your single instance a faster machine or more memory will help more than adding instances that all hit the same LLM API rate limit.

Vertical scaling is also simpler to reason about. One agent, one inbox, one process. No duplicate processing, no coordination, no external state store. If you can solve your throughput problem by upgrading hardware, do that first. Move to horizontal scaling when you hit the ceiling of what a single instance can handle, or when you need fault tolerance that a single process can't provide.

The practical threshold, in my experience, is somewhere around 500 inbound messages per hour per agent. Below that, a single well-tuned instance handles things fine. Above that, you start seeing queue buildup and response latency drift, and that's when horizontal scaling pays for itself.

---