commercetools Product Sync Overview
Author:
Fluent Commerce
Changed on:
31 Jan 2024
Overview
The product synchronization process involves the handling of events and messages through various queues and handlers. It utilizes the commercetools platform for product updates, employs a message router to direct events, and features extensible message handlers for customization.
Key points
- Queue-Based Flow: Events triggered by product initiation or modification in commercetools are pushed to the CT Queue through subscriptions. The CT Queue Listener class reads these messages and forwards them to the Event Queue.
- Message Routing and Handling: The Message Router guides event names from the Event Queue to the relevant handlers. Extensible message handlers offer partners the flexibility to customize logic by overriding existing handlers.
- Batch Job and Delta Sync: The Batch Queue manages bulk data asynchronously, activated by an external scheduler for product catalog synchronization. The Job Router directs the job name to the Product Batch Handler, facilitating filtering and syncing. Automatic Delta Sync ensures continuous background synchronization, relying on commercetools subscriptions.
Technical Details
Product Sync is initiated differently but uses the same flow as described below. Products are added to the external message queue, and a dedicated message connect-sdk consumer will handle the message and export the data to Fluent.
CT Queue: Any product creation/modification will be pushed to ct-queue using commercetools subscriptions. Out-of-the-box(OOTB) connector uses AWS Event Bridge, and it is possible to implement with other queues/subscriptions supported by commercetools. More details on commercetools subscription.
CT Queue Listener: This class reads the message from CT-Queue and pushes it to the Event queue.
Event Queue: The event queue holds the event message to be processed further.
Message Router: Reads the message name from the Event queue and passes it to the appropriate handler. For example, in case of product changes, the message name will be “product“, and the route will be “commercetools.connect.product.upsert“.
Product Updates Handler: This will be a class that reads message data from the internal queue and, based on the product identifier, fetches more information from commercetools-SDK (The core commercetools SDK, which interacts with commercetools APIs ) and maps them with Fluent data model and pushes it to Fluent OMS.
Batch Queue: The Batch queue will handle Bulk data aync. This request will be triggered from an external scheduler with the relevant product catalog sync batch job properties. This request is to fetch products from commercetools API with filters (last updated from - to) and sync with Fluent OMS.
Job Router: The Job router contains the name of the job and passes it to the job handler, e.g.,
`commercetools.connect.batch-product-catalogue-sync`Product Batch Handler: This receives the job name and filters data to fetch data from commercetools SDK and pushes it to the Product Update handler.
The message handlers are fully extensible. Partners can make the changes as per their requirements. The Connect SDK allows message routing with flexible message handlers to process and transform data moving from a source to a destination. Partners can change the message names and override the classes.
Example to extend the functionality
Partners can override existing handlers using the application-connector.yml and Handler class. To change the logic of the Product Upsert handler, first go to
`commercetools-connector/src/main/resources/application-connector.yml`1- route: "commercetools.connect.product.upsert"
2 props:
3 name: "product"
4 inclusion-filter:
5 - "ProductPublished"
6- route: "commercetools.connect.product.deactivate"
7 props:
8 name: "product"
9 inclusion-filter:
10 - "ResourceDeleted"Language: java
Name: Example
Description:
[Warning: empty required content area]New handlers can be added in
`application-connector.yml `Create a new Handler that extends
`MessageHandler``com.fluentcommerce.connect``com.fluentcommerce.connect.custom.commercetools.handler.message.<your-handler>.java`1@Slf4j
2@Component
3@HandlerInfo(name = "FluentProductUpsert", description = "Upsert a product at Fluent OMS")
4public class <your-handler-class> extends MessageHandler {
5 private static final String PRODUCT_ATTRIBUTE = "product";
6 // write your logic
7}Language: java
Name: Example
Description:
[Warning: empty required content area]Batch Job
This option enables product syncing with the help of the Job scheduler. The number of products pushed will depend on the date selected from
`Fluent Settings > Product Catalog Sync`Automatic Delta Sync
This automated synchronization runs in the background via the job scheduler to keep Fluent up to date. This synchronization is based on commercetools subscriptions. Out-of-the-box(OOTB) supported message types are product-published messages, but partners can add more messages. Please refer to commercetools product messages to get more information on CT messages.
Product APIs and Payloads
CT product API
- GET :
`{{host}}/{{project-key}}/products` - GET :
`{{host}}/{{project-key}}/products/{{product-id}}`
Product Message Payload
The Message Payload in CT-queue:
1{
2 "notificationType": "Message",
3 "projectKey": "fluent-ct-dev",
4 "id": "7591e242-a42a-419c-846c-b12e1596442e",
5 "version": 1,
6 "sequenceNumber": 2,
7 "resource": {
8 "typeId": "product",
9 "id": "dbe774ce-86ae-4709-b7e9-dffb8c51fc7c"
10 },
11 "resourceVersion": 2,
12 "resourceUserProvidedIdentifiers": {
13 "slug": {
14 "en": "product_slug_some"
15 }
16 },
17 "type": "ProductPublished",
18 "productProjection": {
19 "id": "dbe774ce-86ae-4709-b7e9-dffb8c51fc7c",
20 "version": 2,
21 "productType": {
22 "typeId": "product-type",
23 "id": "80037be5-c9aa-4f5d-8ef5-0f8c787c7727"
24 },
25 "name": {
26 "en": "Some Product"
27 },
28 "categories": [
29 {
30 "typeId": "category",
31 "id": "17929883-937e-4885-8ff0-37f6770d16e5"
32 }
33 ],
34 "categoryOrderHints": {},
35 "slug": {
36 "en": "product_slug_some"
37 },
38 "masterVariant": {
39 "id": 1,
40 "sku": "SKU-1",
41 "prices": [
42 {
43 "id": "aa7518be-a9c4-4f08-89d1-cf88f5d0a66c",
44 "value": {
45 "type": "centPrecision",
46 "currencyCode": "EUR",
47 "centAmount": 4200,
48 "fractionDigits": 2
49 }
50 }
51 ],
52 "images": [
53 {
54 "url": "http://my.custom.cdn.net/master.png",
55 "label": "Master Image",
56 "dimensions": {
57 "w": 303,
58 "h": 197
59 }
60 }
61 ],
62 "attributes": [],
63 "assets": []
64 },
65 "variants": [
66 {
67 "id": 2,
68 "prices": [],
69 "images": [
70 {
71 "url": "http://my.custom.cdn.net/variant.png",
72 "label": "Variant Image",
73 "dimensions": {
74 "w": 303,
75 "h": 197
76 }
77 }
78 ],
79 "attributes": [],
80 "assets": []
81 }
82 ],
83 "searchKeywords": {},
84 "hasStagedChanges": false,
85 "published": true,
86 "createdAt": "2022-07-18T03:08:24.738Z",
87 "lastModifiedAt": "2022-07-26T10:48:04.028Z"
88 },
89 "removedImageUrls": [],
90 "scope": "All",
91 "createdAt": "2022-07-26T10:48:04.028Z",
92 "lastModifiedAt": "2022-07-26T10:48:04.028Z"
93}Language: json
Name: Example
Description:
[Warning: empty required content area]
