Client State Alerts

client-state related Errors

If you have received an alert related to the client state workflows (cloud functions): . client-state-aggregate-for-customer . client-state-trigger-initializers

See https://github.com/emartech/me-big-query-client-state/#operations for how to regenerate the client_snapshots for a specific customer and date, in case of errors related to client state workflows.

known issues: error in aggregate or trigger for client_snapshots but no customer_id in logs

You can find the cloud function logs using the following gcloud command:

gcloud functions logs read client-state-trigger-initializers \
--project=ems-mobile-engage \
--region=europe-west3 \
--limit=50 \
--start-time="2026-03-31T02:40:00Z" \
--end-time="2026-03-31T03:00:00Z" \
--format="table(severity,timestamp,textPayload)"

Note: change to client-state-aggregate-for-customer for aggregate function and adjust the time range as needed.

If you do find clear customer_ids in the logs for which generation may have failed, verify the latest partition date for the customer’s client_snapshots table and regenerate using the link above (Note that yesterday’s date is the latest partition we should have/want for snapshots).

Should you, however, be unable to find a customer_id in the client_state related cloud functions error logs, you can analyse the latest generated snapshots with the following query(ies):

DECLARE tables ARRAY<STRING>;
DECLARE i INT64 DEFAULT 0;
DECLARE query STRING;

-- Get table list
SET tables = (
SELECT ARRAY_AGG(table_id)
FROM ems-mobile-engage.client_state.__TABLES__
WHERE table_id LIKE 'client_snapshots_%'
);

-- Create temp table for results
CREATE TEMP TABLE partition_results (table_name STRING, max_partition TIMESTAMP);

-- Loop through tables
WHILE i < ARRAY_LENGTH(tables) DO
SET query = FORMAT("""
INSERT INTO partition_results
SELECT '%s', MAX(_PARTITIONTIME)
FROM ems-mobile-engage.client_state.%s
""", tables[OFFSET(i)], tables[OFFSET(i)]);
EXECUTE IMMEDIATE query;
SET i = i + 1;
END WHILE;

-- Show results
SELECT * FROM partition_results ORDER BY max_partition;

This should generate data into a temporary table with the latest partition time for each of the client_snapshots tables. You can then query the temporary table to find out which customer_id (if any) has a latest partition time which is not up to date, which may indicate that the scheduled query to generate the client_snapshots for that customer_id has failed.

SELECT max_partition, count(1) as tables FROM `ems-mobile-engage.<script-run-id>.partition_results` GROUP BY 1 ORDER BY 2 ASC

Expected results should be that all tables have a max_partition time of yesterday’s date. If you see two different max_partition dates, then there is a problem. You will need to regenerate the client_snapshots for these customer_ids as described in the link above.

IMPORTANT NOTE: NULL is to be expected in the results, as this indicates customers which have been deleted. This is because account deletion only deletes all data (truncates) and leaves the actual table behind. These can be safely ignored if you see them in the results of this query.