Distribution Groups

Overview

Distribution Groups (DGs) provide workload isolation in the sending chain. Each DG represents an independent sending pipeline with its own set of Pub/Sub topics and subscriptions. This ensures that high-volume or slow customers do not impact the throughput of others.

Active Distribution Groups

There are currently 5 active distribution groups in production:

DG ID

dg00

Default (all customers not explicitly routed elsewhere)

dg01

Dedicated customers

dg02

Dedicated customers

dg03

Dedicated customers

dg04

Dedicated customers

The default DG is dg00 (configurable via DEFAULT_DISTRIBUTION_GROUP).

Customer Routing

Customers are assigned to distribution groups via the distributionGroups configuration, which maps DG names to arrays of customer IDs. Customers not present in any mapping are routed to the default group (dg00).

Topic Naming Convention

All Pub/Sub topics follow the pattern: sending-<sourceType>-<component>-<dg>

Purpose Pattern Example

TX entry

sending-tx-<dg>

sending-tx-dg00

Batch entry

sending-batch-<dg>

sending-batch-dg01

Fetcher

sending-batch-fetcher-<dg>

sending-batch-fetcher-dg00

Device mapper

sending-<tx|batch>-devicemapper-<dg>

sending-tx-devicemapper-dg02

Personalization

event-pers.push.request-publisher.<tx|batch>.<dg>

event-pers.push.request-publisher.tx.dg00

Delivery (push)

sending-<tx|batch>-push-<provider>-<dg>

sending-batch-push-apple-dg01

Delivery (web push)

sending-<tx|batch>-webpush-vapid-<dg>

sending-tx-webpush-vapid-dg00

Inbox

sending-<tx|batch>-inbox-<dg>

sending-tx-inbox-dg00

Embedded message

sending-<tx|batch>-em-<dg>

sending-batch-em-dg01

Subscriptions follow the pattern: <topic-name>-sub (e.g., sending-tx-dg00-sub)

Deployment

Each DG is deployed as independent Kubernetes replicas. HPA (Horizontal Pod Autoscaler) configurations are maintained per DG in gap/production/.

CLI Argument

All component main.ts files accept the -d <distributionGroup> argument (e.g., -d dg01). BullMQ queues are also suffixed with the DG name (e.g., launches-dg00).

Code Location