Anonymous Contact Deletion for Mobile Engage and Web Push

The anonymous contact deletion collects the contact IDs of anonymous contacts which match a given criteria and then deletes the contacts from the SuiteDB and deletes all devices which are related to those contacts from our devices DB (DynamoDB) in AWS.

An anonymous contact is created by ME for each device where a login is received without contact data (no contact field id and contact field value). In this case a new contact is created in the SuiteDB which has no email, first name and last name (the criteria we apply to ensure a contact is anonymous before we delete it).

The criteria for the selection of the anonymous contacts is the similar as the criteria which is applied on USS ME segments.

Note that contacts deleted from suite this way will appear in the contact_changes table. Which will lead to these being deleted from all Mobile Engage BigQuery sources eventually, as part of the contact-data-deletion.

As the contact deletion now has to potentially support a big amount of customers and apps, a table (anonymous_contacts_configurations) in the push database (PG) has been created to store those configurations. Similarly, a table (web_push_anonymous_contacts_configurations) exists, to store deletion configurations for Web Push apps. The data in these tables is structured in a way so default configs get overridden by more specific configs, up to app-code level granularity. This is to ensure there is always a default fallback config for any app that may be created in the future. These are the type of configs, with the latest fallback last are:

Managing the data in this table can be done through a UI in ems admin. To navigate to this UI you need to select: 'Customer Integrations' > enter the customer you want > find the 'Mobile Engage' integration > Select 'Anonymous contacts config'

Similarly, you can find the Web Push configurations UI under the Web Push integration: 'Customer Integrations' > enter the customer you want > find the 'Web Push' integration > Select 'Anonymous contacts config'

Should you encounter a permissions error on this page, it is highly likely that your admin_id still needs to be added to the whitelist of admins that can access this page in the env of push-notification service. The env var for this is called ADMIN_UI_ACCESS_USERS. You can find your admin_id in the url of the 'Anonymous contacts config' page.

The anon contact deletion cronjob code is deployed as part of me-push-service. It is ran on production every day at 9AM CET for Mobile Engage and at 1PM CET for Web Push respectively. The cronjobs are called:

Once these cronjobs are triggered, they will ingest the whole anonymous_contacts_configurations/web_push_anonymous_contacts_configurations tables in the push PG database. For each customer, it will execute the rules as specified, and queue up the contact_ids in batches, (size configurable using env var ) using Pub/Sub, to be deleted from suite. These deletions are then processed by a separate worker in the me-push-service (me-push-anon-contact-deletion-worker), which sole purpose it is to delete contacts from suite through the Suite API (for both ME and WP).

The number of deleted contacts per customer are logged into LAAS (only stored for a couple of days) and BigQuery (stored indefinitely). You can find the data contatining deletion numbers in BigQuery under the 'anon_contact_deletion' or the 'web_push_anon_contact_deletion' datasets. There’s also a DataStudio report based on the BigQuery tables for both, staging and production environments: staging prod.

There are two possible alarms which can be triggered by the Elastalert monitoring: