Google PubSub (Event Connector) setup for Data Activation

The Data Activation connector for Google Pub/Sub enables you to publish audience and journey data to the Google Cloud Platform (GCP). This integration allows you to stream customer data to GCP for processing, analytics, and downstream system activation.

For additional details, see Google Pub/Sub’s documentation.

For successful setup, make sure you understand the following core concepts of Google Pub/Sub:

• Topic: A named resource to which messages are sent by publishers

• Subscription: A named resource representing the stream of messages from a single, specific topic, to be delivered to the subscribing application

• Message: The combination of data and (optional) attributes that a publisher sends to a topic and is eventually delivered to subscribers

• Message attribute: A key-value pair that a publisher can define for a message (for example, key iana.org/language_tag and value en could be added to messages to mark them as readable by an English-speaking subscriber)

Before you begin

Before configuring the Google Pub/Sub, make sure that you have the following details available:  

• Project ID: ID of the GCP project for the required topic

• Topic: Name of the topic to which the audience/journey data needs to be sent

• API key: With writing access to the topic

• The service account used to generate the API Key requires the PubSub Publisher permission

Instructions

 Step 1: Configure destination

1. On the Data Activation platform, select Connectors in the left-side navigation bar.

2. Select Event connector and click New event connector.

3. Select Google Pub/Sub. The form requires the following information:

   • Name: Display name of the Server to server integration partner, used throughout the Data Activation system (mandatory)

   • Partner number: The partner number, which contains the identifier you would like to receive in Google Pub/Sub

   • API key: API key with writing access to the topic (mandatory)

Step 2: Add the destination connector to an audience or journey

When adding the Google Pub/Sub connector to an audience or journey, there are optional fields to define:

• Source name: Optional. A specific field to understand the source of this data. The "Fill" button will populate the audience name. You can also use team/campaign names

• Property - Value: Optional. Key-value pairs added to the payload for this specific audience/journey step. Useful for sending additional information not related to profile data

Payload structure

Depending on what is defined in the audience/journey and what is added as profile variables, the resulting payload at the topic has the following format:

{
  "profileId": "aaaabbbb-cccc-dddd-eeee-ffffffffffff",
  "sourceId": "dc0beb89-2d63-4ec7-899a-6e986ed8932f",
  "partnerNumber": 33,
  "callType": "NEW",
  "partnerIds": ["4"],
  "profileVars": {
    "lastCategory": "5678"
  },
  "timestamp": 1234567890,
  "siteNumber": 1234,
  "sourceName": "some name",
  "sourceNumber": 123,
  "data": {
    "custom": "variable",
    "another": 123
  }
}

Field descriptions

• profileId: The current profile identifier (partnerNumber 42) for this profile

• sourceId: The unique identifier of the audience/journey step

• partnerNumber: The identifier type selected when creating the connector

• callType: Indicates if the person is newly added, removed, or profileVars have been modified (NEW, MODIFIED, REMOVED)

• partnerIds: Always contains one partnerId. If there are multiple partnerIds on this slot, you will receive multiple messages

• profileVars: The variable values configured to pass to the connector from within the interface

• timestamp: The time in milliseconds since 01-01-1970

• siteNumber: The siteNumber of the specified Data Activation instance

• sourceName: Free format text field to identify the audience in a descriptive manner

• sourceNumber: The numeric ID of an audience. Empty if the message comes from Journey Orchestration

• data: Contains the custom properties defined when creating the audience or journey step

Note

All fields above will always exist but can be empty if they are optional (like profileVars or data).

Error handling

For the Google Pub/Sub connector, Google Cloud-specific error codes are reported. Error documentation can be found in the Google Cloud API documentation.