me-scheduler-service

Table of Contents

Description
Structure
- Segment Execution Interface (SXI)
App Config Variables

Description

The me-scheduler-service does a bit more than simple scheduling although it is one of its central tasks. It can be more seen as a launch-job-production-service because:

it forwards test messages directly to the message generator,
it receives jobs for launches of different types via its REST API,
it triggers the generation of the contact lists at the configured time,
it observes the generation of contact lists and
it creates a launch job after the contact list is ready and forwards it to the respective API endpoint of the message generator.

Structure

A central component of the service is Redis. It is used in conjunction with the BullMQ library to realize

observable job queues
with timer support for any delay and
failure handling with (configurable retry strategies).

NOT YET COMPLETE

Segment Execution Interface (SXI)

The scheduler uses the new Segment Execution Interface.

Figure 1. Sxi Extension Overview

To activate the segmentation process, we must publish a message to a specific topic within the segmentation project. The name of this topic can be found at the following link: https://github.com/emartech/me-scheduler-service/blob/main/src/config/global.ts#L122.

Similarly, the names of the subscriptions can be found at the following link: https://github.com/emartech/me-scheduler-service/blob/main/src/config/global.ts#L152."

App Config Variables

Following you see the configuration variables which are set in GCP for the apps.

Variable Description

Variable	Description
`MAX_JOB_AGE_MS`	The maximum allowed age of a job. Is used to calculate the maximum number of attempts based on a `fixed` retry strategy and the content of `QUEUE_PROCESS_RETRY_DELAY_MS`. Note: Jobs that are too old (calculated max. retry attempt equal 0) will be rejected when inserted into the first stage (this is logged on WARN level)!
`SEGMENT_RUN_RETRY_DELAY_MS`	A delay before we trigger another segment run via the init queue if the current one failed
`QUEUE_PROCESS_RETRY_DELAY_MS`	The delay which is applied between retries if a job fails. This value is used beside `MAX_JOB_AGE_MS` to calculate the maximum allowed retry attempts. So handle with care!
`USE_PRE_RUN_SEGMENTATION`	Flipper: A comma-separated list of customer IDs for which the segment run shall be triggered prior to the launch. The related pre start time is configurable.
`PRE_RUN_SEGMENTATION_OFFSET_MS`	The time a segment run shall be triggered before the planned launch time. Is by default 30 minutes and taken into account if the flipper `USE_PRE_RUN_SEGMENTATION` is enabled for a customer.

MAX_JOB_AGE_MS

The maximum allowed age of a job. Is used to calculate the maximum number of attempts based on a fixed retry strategy and the content of QUEUE_PROCESS_RETRY_DELAY_MS. Note: Jobs that are too old (calculated max. retry attempt equal 0) will be rejected when inserted into the first stage (this is logged on WARN level)!

SEGMENT_RUN_RETRY_DELAY_MS

A delay before we trigger another segment run via the init queue if the current one failed

QUEUE_PROCESS_RETRY_DELAY_MS

The delay which is applied between retries if a job fails. This value is used beside MAX_JOB_AGE_MS to calculate the maximum allowed retry attempts. So handle with care!

USE_PRE_RUN_SEGMENTATION

Flipper: A comma-separated list of customer IDs for which the segment run shall be triggered prior to the launch. The related pre start time is configurable.

PRE_RUN_SEGMENTATION_OFFSET_MS

The time a segment run shall be triggered before the planned launch time. Is by default 30 minutes and taken into account if the flipper USE_PRE_RUN_SEGMENTATION is enabled for a customer.