Msg Generator Service
Components Overview
The sending chain uses Google Pub/Sub exclusively for inter-component messaging. Each component runs as an independent worker consuming from specific Pub/Sub subscriptions, with workloads isolated by Distribution Groups (dg00-dg04).
The pipeline is split into the following stages:
-
API: Receives HTTP requests and publishes to the appropriate Pub/Sub topic
-
Reception: Processes transactional (Aggregator) or batch (Chunker) requests
-
Personalization: Routes messages to the external personalization service or directly to device mapping
-
Device Mapping: Fetches device info, selects content by language, and forwards to delivery
-
Delivery: Provider-specific message formatting and sending (push, web push, inbox, embedded)
For a visual overview, see the Architecture Diagrams.
API
The API part receives HTTP POST requests with campaign data, evaluates the data and publishes
them into the appropriate Pub/Sub topic (sending-tx-<dg> or sending-batch-<dg>).
Chunker (Reception)
The Chunker processes batch campaigns through three sub-components: the Slicer fetches campaign info and creates fetch tasks, the Fetcher retrieves contacts from Suite and publishes per-contact messages, and the Launcher handles throttled campaigns via BullMQ queues.
Aggregator (Reception)
The Aggregator processes transactional send requests from the Web API. It enriches messages with campaign information and routes them to either the external personalization service or directly to the device mapper for non-personalized campaigns.
Personalizer (Personalization)
The Personalizer is an external service (personalization-service). The in-repo code handles routing decisions: determining whether a campaign requires RDS personalization, and publishing to the appropriate topic. The personalizer forwards results to the device mapper.
Device Mapper (Device Mapping)
The Device Mapper fetches device (client) information from PostgreSQL (primary) or DynamoDB (legacy fallback), selects content by language, and forwards push messages to provider-specific delivery topics or the inbox service.
Client Manager
The Client Manager is a subsystem of independent workers responsible for device data lifecycle
management across multi-tenant PostgreSQL schemas (tenant_<customerId>). Sub-workers handle
device writes, push token status, contact/account deletion, tenant activation, orphan cleanup,
and dashboard metrics collection.
Internal Modules
Distribution Groups
The message generator supports distribution groups (dg00-dg04), where each distribution group has its own independent sending chain. For details see Distribution Groups.
Monitoring
For information about the monitoring of batch and transactional campaigns please refer to the related page.