Documentation

Subscriptions for Messages and Notifications

Subscriptions allow you to be notified of new messages or changes via a Message Queue of your choice

Subscriptions are used to trigger an asynchronous background process in response to an event on the commercetools platform. Common use cases include sending an Order Confirmation Email, charging a Credit Card after the delivery has been made, or synchronizing customer accounts to a Customer Relationship Management (CRM) system.

As a payload, a Subscription delivers one of the predefined Messages or any Change to a resource.

The first part of this document explains how to setup a subscription, while the second part details the payload and delivery guarantees.

Representations

Subscription

SubscriptionDraft

  • key - String - Optional - User-specific unique identifier for the subscription
  • destination - Destination - The Message Queue into which the notifications are to be sent
  • messages - Array of MessageSubscription - Optional - The messages to be subscribed to.
  • changes - Array of ChangeSubscription - Optional - The change notifications to be subscribed to.
  • format - Format - Optional - The format in which the payload is delivered.

Either messages or changes need to be set.

Destination

A destination contains all info necessary for the commercetools platform to deliver a message onto your Message Queue. Message Queues can be differentiated by the type field.

Currently AWS SQS, AWS SNS, AWS EventBridge, Azure Service Bus, Azure Event Grid and Google Cloud Pub/Sub are supported.

When retrieving a Destination, secrets, or access keys are partially hidden for security reasons.

SQS Destination

AWS SQS is a pull-queue on AWS. We support the Standard queue type (the FIFO queue type is not supported). If your use-case requires a FIFO queue, consider using [EventBrige] (#eventbridge-destination).

  • type - String - "SQS"
  • queueUrl - String
  • accessKey - String
  • accessSecret - String - Partially hidden on retrieval
  • region - String

The queue must exist beforehand. The MaximumMessageSize of the queue must be set to the default value of 262,144 (256 KiB). We recommend creating an Identity and Access Management (IAM) user with an accessKey and accessSecret pair specifically for each subscription. The IAM user should only have the sqs:SendMessage permission on this queue.

SNS Destination

AWS SNS can be used to push messages to AWS Lambda, HTTP endpoints (webhooks) or fan-out messages to SQS queues.

  • type - String - "SNS"
  • topicArn - String
  • accessKey - String
  • accessSecret - String - Partially hidden on retrieval

The topic must exist beforehand. We recommend creating an IAM user with an accessKey and accessSecret pair specifically for each subscription. The IAM user should only have the sns:Publish permission on this topic.

EventBridge Destination

AWS EventBridge can be used to push events and messages to a serverless event bus that can forward them to AWS SQS, SNS, Lambda, and other AWS services based on forwarding rules.

  • type - String - "EventBridge"
  • region - String
  • accountId - String

The accountId is the ID of the AWS account that will receive events from commercetools. Events can be sent to custom regions by providing the region in the destination payload. At this point, no additional authentication is required for the subscription destination. However, once the destination is successfully created in the commercetools platform, an equivalent "partner event source" will be created in AWS EventBridge. This event source has to be associated with an event bus for the subscription set up to be complete. Learn how to integrate EventBridge with the commercetools platform by following our EventBridge tutorial.

Azure Service Bus Destination

Azure Service Bus can be used as a pull-queue with Queues or to fan-out messages with Topics and Subscriptions.

  • type - String - "AzureServiceBus"
  • connectionString - String - The SharedAccessKey is partially hidden on retrieval.

To setup a subscription with Azure Service Bus you first need to create a Queue/Topic in the Azure Portal. To grant access to resources Azure Service Bus uses the concept of Shared Access Policies. A shared access policy is defined on a scope and includes certain permissions. To allow the commercetools platform to push messages to your Queue/Topic you need to create a Shared Access Policy on your Queue/Topic and include the Send permission. When you open the policy in the Azure Portal you will find that Azure generated two so called connection strings for your policy. A connection string might look like this:

Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=<key-name>;SharedAccessKey=<key>;EntityPath=<queue-name>

You can pass either of the two connection strings as argument for the connectionString parameter of an Azure Service Bus Destination.
Make sure that the entity path points to your Queue/Topic.

Azure Event Grid Destination

Azure Event Grid can be used to push messages to Azure Functions, HTTP endpoints (webhooks), and several other Azure tools.

Event Grid can only be used with the CloudEvents Format.

  • type - String - "EventGrid"
  • uri - String - The URI of the Topic
  • accessKey - String - Partially hidden on retrieval

To setup a subscription with Azure Event Grid you first need to create a Topic in the Azure Portal. To allow the commercetools platform to push messages to your Topic, you need to provide an access key. These can also be found in the Azure Portal after creating the Topic in the section Access Keys.

Google Cloud Pub/Sub Destination

Google Cloud Pub/Sub can be used both as a pull-queue, and to push messages to for example Google Cloud Functions or HTTP endpoints (webhooks).

  • type - String - "GoogleCloudPubSub"
  • projectId - String - The ID of the project that contains the Pub/Sub topic
  • topic - String - Name of the topic

The topic must exist beforehand. The topic has to give the pubsub.topics.publish permission to the service account subscriptions@commercetools-platform.iam.gserviceaccount.com.

If used with the CloudEvents Format, the message conforms to the PubSub Protocol Binding of the Structured Content Mode.

MessageSubscription

  • resourceTypeId - String
  • types - Array of String - Optional

types must contain valid message types for this resource, for example for resource type product the message type ProductPublished is valid. If no types of messages are given, the subscription is valid for all messages of this resource.

For supported resources and message types, please refer to the Messages Types Documentation. Messages will be delivered even if the Messages Query HTTP API is deactivated.

The Message Subscription Payload is delivered.

ChangeSubscription

  • resourceTypeId - String

The following list of resourceTypeId are currently supported:

  • cart
  • cart-discount
  • category
  • channel
  • customer
  • customer-email-token
  • customer-group
  • customer-password-token
  • discount-code
  • extension
  • inventory-entry
  • key-value-document - Custom Objects
  • order - Changes to Orders via Order Edits do not trigger a Message. To trigger a Message, subscription to OrderEditAppliedMessage is necessary.
  • order-edit
  • payment
  • product
  • product-discount
  • product-selection BETA
  • product-type
  • review
  • shipping-method
  • shopping-list
  • state
  • store
  • subscription
  • tax-category
  • type
  • zone

Different payloads for resource creation, update, and deletion are delivered.

Format

The format in which the payload is delivered.

Platform Format

The Platform Format uses constructs that are similar to the ones used in the REST API, for example on the Messages Query HTTP API

  • type - String - "Platform"

CloudEvents Format BETA

CloudEvents is a specification for describing event data in a common way. CloudEvents seeks to ease event declaration and delivery across services, platforms, and beyond. It is hosted by the Cloud Native Computing Foundation.

The CloudEvents format can be used with any Destination, and the payload delivered in the JSON Event Format. Azure Event Grid offers native support to filter and route CloudEvents.

We offer long-term support for version 1.0. The version 0.1 is deprecated and will not be supported after March 2020.

  • type - String - "CloudEvents"
  • cloudEventsVersion - String - "1.0"

Subscription Health Status

Values of the SubscriptionHealthStatus enumeration:

  • Healthy - The subscription is delivering messages as expected
  • ConfigurationError - Messages can not be delivered with the current configuration. If the configuration is fixed in this health status, all messages that haven't been delivered yet will still be delivered and the status will change to Healthy.
  • ConfigurationErrorDeliveryStopped - Messages can not be delivered with the current configuration and delivery of those is not attempted anymore. Undelivered messages are not retained and will not be delivered if the configuration is fixed in this status, but the status will change to Healthy as soon as new messages can be delivered successfully again.
  • TemporaryError - Messages can not be delivered currently, but not due to the configuration. For example the destination has a temporary outage.

Common causes for ConfigurationErrors are deleting the destination queue, deleting the access credentials, or removing the permission to push messages into the queue.

ConfigurationErrors are automatically turned into ConfigurationErrorDeliveryStopped after some time. See the Delivery Guarantees section for more details.

The configuration can be fixed for example by re-creating the configuration on the destination side, or by setting a new configuration with the update action Change Destination.

Please note that the health of the delivery infrastructure is independent of the SubscriptionHealthStatus, and can be checked on the status page (it is part of the Background Services).

Get a Subscription

Get a Subscription by ID

Retrieves the representation of a subscription by its id.

Endpoint: /{projectKey}/subscriptions/{id}
Method: GET
OAuth 2.0 Scopes: manage_subscriptions:{projectKey}
Response Representation: Subscription

Get a Subscription by Key

Retrieves the representation of a subscription by its key.

Endpoint: /{projectKey}/subscriptions/key={key}
Method: GET
OAuth 2.0 Scopes: manage_subscriptions:{projectKey}
Response Representation: Subscription

Get the Health Status of a Subscription by ID

Retrieves the SubscriptionHealthStatus of a subscription by its id. This endpoint can be polled by a monitoring or alerting system, to automatically inform you if your configuration is not working anymore, or in case of any other error.

Endpoint: /{projectKey}/subscriptions/{id}/health
Method: GET

This endpoint is publicly visible.

No Body is returned. The SubscriptionHealthStatus is mapped onto HTTP status codes as follows:

  • Healthy - 200
  • ConfigurationError and ConfigurationErrorDeliveryStopped - 400
  • TemporaryError - 503

Query Subscriptions

Endpoint: /{projectKey}/subscriptions
Method: GET
OAuth 2.0 Scopes: manage_subscriptions:{projectKey}
Response Representation: PagedQueryResult with results containing an array of Subscription
Query Parameters:

Create a Subscription

Endpoint: /{projectKey}/subscriptions
Method: POST
OAuth 2.0 Scopes: manage_subscriptions:{projectKey}
Request Representation: SubscriptionDraft
Response Representation: Subscription

The creation of a Subscription is eventually consistent, it may take up to a minute before it becomes fully active.

To test that the destination is correctly configured, a test message will be put into the queue. If the message could not be delivered, the subscription will not be created. The payload of the test message is a notification of type ResourceCreated for the resourceTypeId subscription.

Currently, a maximum of 50 Subscriptions can be created per project.

Update Subscription

Updates of a Subscription are eventually consistent, it may take up to a minute before changes becomes fully active.

Update Subscription by ID

Endpoint: /{projectKey}/subscriptions/{id}
Method: POST
OAuth 2.0 Scopes: manage_subscriptions:{projectKey}
Response Representation: Subscription
Fields:

  • version - Number - Required
    The expected version of the subscription on which the changes should be applied. If the expected version does not match the actual version, a 409 Conflict will be returned.
  • actions - Array of UpdateAction - Required
    The list of update actions to be performed on the subscription.

Update Subscription by Key

Endpoint: /{projectKey}/subscriptions/key={key}
Method: POST
OAuth 2.0 Scopes: manage_subscriptions:{projectKey}
Response Representation: Subscription
Fields:

  • version - Number - Required
    The expected version of the subscription on which the changes should be applied. If the expected version does not match the actual version, a 409 Conflict will be returned.
  • actions - Array of UpdateAction - Required
    The list of update actions to be performed on the subscription.

Update actions

Set Key

  • action - String - "setKey"
  • key - String - Optional
    If key is absent or null, the existing key, if any, will be removed.

Set Messages

  • action - String - "setMessages"
  • messages - Array of MessageSubscription - Optional - The messages to be subscribed to.

messages can only be unset if changes is set.

Set Changes

  • action - String - "setChanges"
  • changes - Array of ChangeSubscription - Optional - The change notifications to be subscribed to.

changes can only be unset if messages is set.

Change Destination

  • action - String - "changeDestination"
  • destination - Destination

Updating the destination is eventually consistent, it may take up to a minute before it becomes fully active. During this time, messages may be delivered simultaneously to both the old and the new destination.

To test that the destination is correctly configured, a test message will be put into the queue. If the message could not be delivered, the update will fail. The payload of the test message is a notification of type ResourceCreated for the resourceTypeId subscription. The status will change to Healthy, if it wasn't already.

Delete Subscription

The deletion of a Subscription is eventually consistent, it may take up to a minute before it becomes fully deactived.

Delete Subscription by ID

Endpoint: /{projectKey}/subscriptions/{id}
Method: DELETE
OAuth 2.0 Scopes: manage_subscriptions:{projectKey}
Query Parameters:

  • version - Number - Required

Delete Subscription by Key

Endpoint: /{projectKey}/subscriptions/key={key}
Method: DELETE
OAuth 2.0 Scopes: manage_subscriptions:{projectKey}
Query Parameters:

  • version - Number - Required

Delivery

The Delivery Payload depends on the chosen Format.

Delivery Payloads for the Platform Format

All payloads for the Platform Format share these common fields:

  • projectKey - String - The key of the project. Useful if the same queue is filled from multiple projects.
  • notificationType - String - Identifies the payload
  • resource - Reference to an object
    A reference to the resource that triggered this delivery.
  • resourceUserProvidedIdentifiers - UserProvidedIdentifiers - Optional

Message Subscription Payload

This payload will be sent for a MessageSubscription.

The payload will always contain these common fields for any message:

If the payload fits into the size limit of your message queue (the limit is often 256kb), all additional fields for the specific message are included as well (along with the type field). If the payload did not fit, it can be retrieved from the Messages Query HTTP API if messages are enabled.

If the payload does not fit into the size limit of your message queue, the field payloadNotIncluded is present.

PayloadNotIncluded

  • reason - String
  • payloadType - String - The value of the type field in the original payload.

ResourceCreated Payload

This payload will be sent for a ChangeSubscription if a resource was created.

  • notificationType - String - "ResourceCreated"
  • version - Number - The version of the resource after it was created
  • modifiedAt - DateTime - The lastModifiedAt of the resource after it was created

ResourceUpdated Payload

This payload will be sent for a ChangeSubscription if a resource was updated. Note: this payload is also sent if a resource is updated by an automatic platform task, like a change in product availability.

  • notificationType - String - "ResourceUpdated"
  • version - Number - The version of the resource after the update
  • oldVersion - Number - The version of the resource before the update
  • modifiedAt - DateTime - The lastModifiedAt of the resource after it was updated

ResourceDeleted Payload

This payload will be sent for a ChangeSubscription if a resource was deleted.

  • notificationType - String - "ResourceDeleted"
  • version - Number - The version of the resource at the time of deletion
  • modifiedAt - DateTime - The point in time when the resource was deleted
  • dataErasure - Boolean - Optional - true if the dataErasure parameter was set to true.

Delivery Payload for the CloudEvents Format

The CloudEvents Format wraps the existing payloads into a standardized envelope. The payload can be found inside the data field.

  • specversion - String
  • id - String
  • type - String
    The type is namespaced with com.commercetools, followed by the Resource Type ID and the type of subscription (either message or change).
    The final entry for a MessageSubscription is the Messages Type, for example com.commercetools.product.message.ProductPublished.
    The final entry for a ChangeSubscription is either ResourceCreated, ResourceUpdated or ResourceDeleted, for example com.commercetools.order.change.ResourceCreated.
  • source - String - The default REST URI of the Resource Type ID that triggered this event, including the project key. For example /my-project/products/4aa81de5-45d3-4fca-96e2-860df4389d09.
  • subject - String - The ID of the resource that triggered this event, for example 4aa81de5-45d3-4fca-96e2-860df4389d09
  • time - DateTime - Corresponds to the lastModifiedAt of the resource at the time the event was triggered.
  • sequence - String - Optional - Corresponds to the sequenceNumber of a MessageSubscription. Can be used to sort a stream of events.
  • sequencetype - String - Optional - "Integer"
  • dataref - String - Optional - Only set for MessageSubscriptions. The URI from which the message can be retrieved if messages are enabled.
  • data - Object - Either a Message Payload, a ResourceCreated Payload, a ResourceUpdated Payload, or a ResourceDeleted Payload.

Delivery Guarantees

At-least-once delivery

If the commercetools platform did not get a positive acknowledgement that messages have been accepted by the destination, it will retry to deliver the messages. Our Retry Policy depends on the Subscription Health Status:

  • TemporaryError- Retry for up to 48 hours, afterwards messages may be dropped.
  • ConfigurationError - Retry for up to 24 hours (for production projects) or up to 1 hour (development or staging projects), afterwards the status changes to ConfigurationErrorDeliveryStopped and messages are dropped.

A side effect of the retry is that the same message may be put several times into the message queue. An idempotent message processor that doesn't process the same message twice can check whether the message was already processed. For notificationType "Message" use the fields resource.id and sequenceNumber, or for other types the fields resource.id and version.

We guarantee that all message payloads we try to deliver to a destination are valid according to its specification. Therefore, the only cases where messages can not be delivered are due to an ongoing incident, or due to a misconfiguration.

You can monitor the health of a subscription with a tool of your choice. For production-critical queues, we recommend to set up an automatic alert.

No guarantee on order

Messages are not guaranteed to be delivered in their natural order (for example with ascending sequenceNumber or ascending version). This is especially true in the case of retries.

For notificationType "Message", a message processor can use the fields resource.id and sequenceNumber to process messages in the correct order (for example if the last processed sequenceNumber was 2, and the current message is 4, the current message can be put back into the queue for processing at a later point in time).

For other notificationType, the fields resource.id, version and (in case of update) oldVersion can be used. Note that version is not sequential.

No guarantee on delivery time

Messages are not guaranteed to be delivered within a certain time frame. Although most messages are delivered within seconds, delays of several minutes can be caused by a variety of factors, such as platform scaling. Therefore we do not recommend using subscriptions for time-critical use-cases where a few minutes delay is seen as an incident.