Fluent Commerce Logo
Docs
Sign In

commercetools Product Sync Overview

Essential knowledge

Author:

Fluent Commerce staff

Changed on:

31 Jan 2024

Overview

The product synchronization process involves the handling of events and messages through various queues and handlers. It utilizes the commercetools platform for product updates, employs a message router to direct events, and features extensible message handlers for customization.

Key points

  • Queue-Based Flow: Events triggered by product initiation or modification in commercetools are pushed to the CT Queue through subscriptions. The CT Queue Listener class reads these messages and forwards them to the Event Queue.
  • Message Routing and Handling: The Message Router guides event names from the Event Queue to the relevant handlers. Extensible message handlers offer partners the flexibility to customize logic by overriding existing handlers.
  • Batch Job and Delta Sync: The Batch Queue manages bulk data asynchronously, activated by an external scheduler for product catalog synchronization. The Job Router directs the job name to the Product Batch Handler, facilitating filtering and syncing. Automatic Delta Sync ensures continuous background synchronization, relying on commercetools subscriptions.

Technical Details

Product Sync is initiated differently but uses the same flow as described below. Products are added to the external message queue, and a dedicated message connect-sdk consumer will handle the message and export the data to Fluent.

No alt provided

CT Queue: Any product creation/modification will be pushed to ct-queue using commercetools subscriptions. Out-of-the-box(OOTB) connector uses AWS Event Bridge, and it is possible to implement with other queues/subscriptions supported by commercetools. More details on commercetools subscription.

CT Queue Listener: This class reads the message from CT-Queue and pushes it to the Event queue.

Event Queue: The event queue holds the event message to be processed further.

Message Router: Reads the message name from the Event queue and passes it to the appropriate handler. For example, in case of product changes, the message name will be “product“, and the route will be “commercetools.connect.product.upsert“.

Product Updates Handler: This will be a class that reads message data from the internal queue and, based on the product identifier, fetches more information from commercetools-SDK (The core commercetools SDK, which interacts with commercetools APIs ) and maps them with Fluent data model and pushes it to Fluent OMS.

Batch Queue: The Batch queue will handle Bulk data aync. This request will be triggered from an external scheduler with the relevant product catalog sync batch job properties. This request is to fetch products from commercetools API with filters (last updated from - to) and sync with Fluent OMS.

Job Router: The Job router contains the name of the job and passes it to the job handler, e.g.,

`commercetools.connect.batch-product-catalogue-sync`
.

Product Batch Handler: This receives the job name and filters data to fetch data from commercetools SDK and pushes it to the Product Update handler.

The message handlers are fully extensible. Partners can make the changes as per their requirements. The Connect SDK allows message routing with flexible message handlers to process and transform data moving from a source to a destination. Partners can change the message names and override the classes.

Example to extend the functionality

Partners can override existing handlers using the application-connector.yml and Handler class. To change the logic of the Product Upsert handler, first go to

`commercetools-connector/src/main/resources/application-connector.yml`
and replace the "handler" under routes > name > handler node.

1- route: "commercetools.connect.product.upsert"
2     props:
3       name: "product"
4       inclusion-filter:
5          - "ProductPublished"
6- route: "commercetools.connect.product.deactivate"
7     props:
8       name: "product"
9       inclusion-filter:
10          - "ResourceDeleted"

Language: java

Name: Example

Description:

[Warning: empty required content area]

New handlers can be added in 

`application-connector.yml `
specific to CT message types for the product entity.

Create a new Handler that extends 

`MessageHandler`
 and ensure that your package is under the main SDK package 
`com.fluentcommerce.connect`
, for example, 
`com.fluentcommerce.connect.custom.commercetools.handler.message.<your-handler>.java`
. For more details on how to extend, please refer to the extend guidelines.

1@Slf4j
2@Component
3@HandlerInfo(name = "FluentProductUpsert", description = "Upsert a product at Fluent OMS")
4public class <your-handler-class> extends MessageHandler {
5    private static final String PRODUCT_ATTRIBUTE = "product";
6    // write your logic
7}

Language: java

Name: Example

Description:

[Warning: empty required content area]

Batch Job

This option enables product syncing with the help of the Job scheduler. The number of products pushed will depend on the date selected from 

`Fluent Settings > Product Catalog Sync`
. The extension will look for products that have been modified from the date specified until the synchronization starts running. It implicitly adds the categories mapped against the products.

Automatic Delta Sync

This automated synchronization runs in the background via the job scheduler to keep Fluent up to date. This synchronization is based on commercetools subscriptions. Out-of-the-box(OOTB) supported message types are product-published messages, but partners can add more messages. Please refer to commercetools product messages to get more information on CT messages.

Product APIs and Payloads

CT product API
  • GET :
    `{{host}}/{{project-key}}/products`
  • GET :
    `{{host}}/{{project-key}}/products/{{product-id}}`
Product Message Payload

The Message Payload in CT-queue:

1{
2      "notificationType": "Message",
3      "projectKey": "fluent-ct-dev",
4      "id": "7591e242-a42a-419c-846c-b12e1596442e",
5      "version": 1,
6      "sequenceNumber": 2,
7      "resource": {
8        "typeId": "product",
9        "id": "dbe774ce-86ae-4709-b7e9-dffb8c51fc7c"
10      },
11      "resourceVersion": 2,
12      "resourceUserProvidedIdentifiers": {
13        "slug": {
14          "en": "product_slug_some"
15        }
16      },
17      "type": "ProductPublished",
18      "productProjection": {
19        "id": "dbe774ce-86ae-4709-b7e9-dffb8c51fc7c",
20        "version": 2,
21        "productType": {
22          "typeId": "product-type",
23          "id": "80037be5-c9aa-4f5d-8ef5-0f8c787c7727"
24        },
25        "name": {
26          "en": "Some Product"
27        },
28        "categories": [
29          {
30            "typeId": "category",
31            "id": "17929883-937e-4885-8ff0-37f6770d16e5"
32          }
33        ],
34        "categoryOrderHints": {},
35        "slug": {
36          "en": "product_slug_some"
37        },
38        "masterVariant": {
39          "id": 1,
40          "sku": "SKU-1",
41          "prices": [
42            {
43              "id": "aa7518be-a9c4-4f08-89d1-cf88f5d0a66c",
44              "value": {
45                "type": "centPrecision",
46                "currencyCode": "EUR",
47                "centAmount": 4200,
48                "fractionDigits": 2
49              }
50            }
51          ],
52          "images": [
53            {
54              "url": "http://my.custom.cdn.net/master.png",
55              "label": "Master Image",
56              "dimensions": {
57                "w": 303,
58                "h": 197
59              }
60            }
61          ],
62          "attributes": [],
63          "assets": []
64        },
65        "variants": [
66          {
67            "id": 2,
68            "prices": [],
69            "images": [
70              {
71                "url": "http://my.custom.cdn.net/variant.png",
72                "label": "Variant Image",
73                "dimensions": {
74                  "w": 303,
75                  "h": 197
76                }
77              }
78            ],
79            "attributes": [],
80            "assets": []
81          }
82        ],
83        "searchKeywords": {},
84        "hasStagedChanges": false,
85        "published": true,
86        "createdAt": "2022-07-18T03:08:24.738Z",
87        "lastModifiedAt": "2022-07-26T10:48:04.028Z"
88      },
89      "removedImageUrls": [],
90      "scope": "All",
91      "createdAt": "2022-07-26T10:48:04.028Z",
92      "lastModifiedAt": "2022-07-26T10:48:04.028Z"
93}

Language: json

Name: Example

Description:

[Warning: empty required content area]
Fluent Commerce staff

Fluent Commerce staff

Copyright © 2024 Fluent Retail Pty Ltd (trading as Fluent Commerce). All rights reserved. No materials on this docs.fluentcommerce.com site may be used in any way and/or for any purpose without prior written authorisation from Fluent Commerce. Current customers and partners shall use these materials strictly in accordance with the terms and conditions of their written agreements with Fluent Commerce or its affiliates.

Fluent Logo