commercetools Connector Extend Configuration
Essential knowledge
Intended Audience:
Technical User
Author:
Girish Padmanabha
Changed on:
17 June 2026
Overview
The commercetools Connector features a highly adaptable runtime architecture governed by four baseline configuration files:`application-connector.yml`, `application.yml`, `bootstrap.yml`, and `logback-spring.xml`. Together, these files define core connector settings, server/container properties, and logging rules.To manage multi-environment rollouts efficiently, you can override common baseline settings by creating dedicated files that follow standard Spring Boot profile conventions (e.g., `application-dev.yml`) to isolate your development, staging, and production environments.Key points
- What You Will Learn: You will understand how to safely configure, secure, and scale the bidirectional data flow between commercetools and Fluent Commerce across multi-profile cloud environments without modifying core connector code.
- Core File Architecture: The connector requires a strict baseline of four configuration files (
`application-connector.yml`,`application.yml`,`bootstrap.yml`, and`logback-spring.xml`). You can dynamically override any environment setting by appending standard Spring Boot profile suffixes (e.g.,`application-dev.yml`). - Critical Operational Guardrails: * Job Serialization: The asynchronous job scheduler enforces strict concurrency control—multiple instances of the same data synchronization job cannot run simultaneously.
- Data Retention: Manual
`customDateRange`overrides are destructive parameters; the connector executes the targeted historical timeframe exactly once and automatically deletes the custom range to prevent looping data.
- Data Retention: Manual
- Security and Performance Impact: Moving or updating sensitive external platform endpoints requires configuring the connector's dedicated credential storage layer. Additionally, message processing throughout during peak traffic events is directly governed by adjusting the
`pool-size`and queue listener properties.
Core Manifest Hierarchy
The connector runtime checks and parses a minimum layout of four configuration files during boot operations. Environment-specific overrides follow standard SpringBoot multi-profile layout conventions (e.g., appending the profile indicator to create target configurations like`application-dev.yml`).`application-connector.yml`: Manages project and connector-specific route architectures and operational definitions.`application.yml`: Declares core defaults, system variables, and global initialization baselines.`bootstrap.yml`: Governs container layouts and configuration parameters matching specific server deployment contexts.`logback-spring.xml`: Organizes logging appenders, metrics collection boundaries, and environmental log formatting rules.
Security & Secrets Blueprint
To connect securely to external engines, credential sets must reside within a verified, restricted vault location. Core service customized bindings can be registered via the SDK Customization Manual, while configuration practices follow the standard Credential / Secret Setup Guide.Ingress Listener Typologies
Global pipeline orchestration depends directly on top-level infrastructure setting arrays. External webhooks run independently of these configuration criteria.Core Settings Config Matrix
| Property | Required | Description |
`message-listener-mode` | Yes | Defines the listener layout assigned exclusively to the connector message queue. |
`batch-listener-mode` | Yes | Sets the ingestion layer properties used for the structural connector batch queue. |
`available-listeners` | Yes | Enumerates the complete array of target tracking listeners recognized by the running service instance. |
1fluent-connect:
2 message-listener-mode: sqs
3 batch-listener-mode: sqs
4 available-listeners:
5 - "sqs"
6 - "kafka"Individual Listener Allocation Blocks
Instances support any number of parallel processing threads running simultaneously. Each definition registers unique properties matching the structure below:1listeners:
2 # listener configurion block
3 listener-id:
4 name: "ENV_VARIABLE"
5 pool-size: 20
6 prefix: "my-prefix"
7 shutdown-timeout: 300
8 visibility-timeout: 120
9 poll-wait: 20
10 dlq-suffix: "-dlq"
11 retry: 5
12 type: sqs | Property | Required | Default Value | Description |
`listener-id` | Yes | n/a | Structural identifier of the listener matching codes configured via `@ListenerInfo`. |
`name` | Yes | Empty String | Environmental variable mapping containing the physical target queue identifier. |
`type` | Yes | NOT_DEFINED | Structural queuing type selected from supported platform listeners. |
`fluent-account` | Yes | n/a | Directing target account reference used to isolate account execution. |
`pool-size` | No | 10 | Thread count parameters governing parallel message processing capacity. |
`prefix` | No | default | Formatting tag assigned to execution threads for traceability. |
`shutdown-timeout` | No | 300 | Seconds allowed for active tasks to process prior to container termination commands. |
`visibility-timeout` | No | 120 | Invisibility window in seconds ensuring unique delivery while a node executes a task. |
`poll-wait` | No | 10 | Timing metrics in seconds indicating how long a listener waits to pull items from queues. |
`dlq-suffix` | No | -dlq | Formatting suffix appended to establish Dead Letter Queue designations. |
`retry` | No | 2 | Count defining queue retry limits prior to rerouting messages into the target DLQ. |
`max-allowed-pull-size` | No | 10 | Upper bounds limiting maximum message count per queue pull request. |
`props` | No | n/a | Extensible map layout storing custom structural parameters. |
Engine Chronological Orchestration (Job Scheduler)
The application delegates tracking timelines to external workflow management networks. Tasks trigger asynchronously via structured REST invocations, instantly processing execution parameters before providing standard`200 OK` return codes.Job Ingestion Rest Endpoints
1PUT https://<domain>/api/v1/fluent-connect/scheduler/add/<fluent-account>/<fluent-retailer>/<job-name>`fluent-account`: Fluent infrastructure account reference.`fluent-retailer`: Isolated context identifier tracking retailer context.`job-name`: Assigned handler class targeting the batch operation.
1PUT https://localhost:8080/api/v1/fluent-connect/scheduler/add/cnctdev/1/batch-inventory-syncPre- and Post-Execution Lifecycle Rules
Every job executing inside the connector environment complies uniformly with the following structural rules:- Concurrency blocks ensure duplicate instances cannot run simultaneously.
- Context models extract runtime configurations directly out of active Fluent settings.
- Logical modules compute bounded date ranges to isolate target execution deltas.
- Core handler logic runs the assigned integration process in UTC time.
- Post-execution routines persist final runtime parameters for subsequent calculations.
Configuration Management Classes
System jobs balance parameters between two discrete layers:- Build Profiles: Implemented directly as local codebase variables within deployable manifests (e.g., properties maps inside YAML routing lists).
- Runtime Settings: Managed dynamically as state properties objects stored inside Fluent settings. Keys compile structurally using prefix concatenation parameters:
YAML
job-scheduler:`config-prefix:
`` `"fc.connect.<connector-name>.data-sync"`
`
`batch-inventory-sync` maps to configuration key `fc.connect.<connector-name>.data-sync.batch-inventory-sync`.Delta Tracking Chronological Calculus
Every engine loop receives computed timestamp bounds via its job context, allowing delta-mode updates. If`previousEndDate` parameter inputs are missing or unassigned, start markers default to Unix Epoch time.- Scenario A (First Run, No Baseline Date): From: Epoch time $\rightarrow$ To: Now.
- Scenario B (First Run, Baseline Date Present): From: Explicit Date $\rightarrow$ To: Now.
- Scenario C (Subsequent Cycles): From: Preceding End DateTime (
`previousEndDate`) $\rightarrow$ To: Now.
Configuration Profiles & State Traces
Baseline Job Profile Setup (Unexecuted State)
1{
2 "previousEndDate": "",
3 "props": {
4 "fluentNetworkRef": "BASE_67"
5 }
6}First Execution State Footprint
1{
2 "previousEndDate": "2022-09-22T03:00:00",
3 "props": {
4 "fluentNetworkRef": "BASE_67"
5 },
6 "lastRun": {
7 "param": {
8 "start": "1970-01-01T00:00:00",
9 "end": "2022-09-22T03:00:00",
10 "props": {
11 "fluentNetworkRef": "BASE_67"
12 }
13 },
14 "jobStart": "2022-09-22T03:00:00",
15 "jobsEnd": "2022-09-22T03:05:00",
16 "status": "SUCCESSFUL"
17 }
18}Subsequent Execution State Footprint
1{
2 "previousEndDate": "2022-09-23T02:00:00",
3 "props": {
4 "fluentNetworkRef": "BASE_67"
5 },
6 "lastRun": {
7 "param": {
8 "start": "2022-09-22T03:00:00",
9 "end": "2022-09-23T02:00:00",
10 "props": {
11 "fluentNetworkRef": "BASE_67"
12 }
13 },
14 "jobStart": "2022-09-23T02:00:00",
15 "jobsEnd": "2022-09-23T02:05:00",
16 "status": "SUCCESSFUL"
17 }
18}User-Defined Overrides State Footprint
Users can manually inject single-use chronological constraints. The connector applies these properties for one run cycle before removing the object properties.1{
2 "customDateRange": {
3 "start": "2022-04-22T00:00:00",
4 "end": "2022-04-23T00:00:00"
5 },
6 "previousEndDate": "2022-09-23T02:00:00",
7 "props": {
8 "fluentNetworkRef": "BASE_67"
9 },
10 "lastRun": {
11 "param": {
12 "start": "2022-09-22T03:00:00",
13 "end": "2022-09-23T02:00:00",
14 "props": {
15 "fluentNetworkRef": "BASE_67"
16 }
17 },
18 "jobStart": "2022-09-23T02:00:00",
19 "jobsEnd": "2022-09-23T02:05:00",
20 "status": "SUCCESSFUL"
21 }
22}Automatic Resume State Footprint (Post-Override Cleanup)
1{
2 "previousEndDate": "2022-09-23T02:00:00",
3 "props": {
4 "fluentNetworkRef": "BASE_67"
5 },
6 "lastRun": {
7 "param": {
8 "start": "2022-04-22T00:00:00",
9 "end": "2022-04-23T00:00:00",
10 "props": {
11 "fluentNetworkRef": "BASE_67"
12 }
13 },
14 "jobStart": "2022-09-24T02:00:00",
15 "jobsEnd": "2022-09-24T02:05:00",
16 "status": "SUCCESSFUL"
17 }
18}In-Flight Lock Tracking Footprint
1{
2 "previousEndDate": "2022-09-23T02:00:00",
3 "executionStart" : "2022-09-24T02:00:00",
4 "props": {
5 "fluentNetworkRef": "BASE_67"
6 },
7 "lastRun": {
8 "param": {
9 "start": "2022-04-22T00:00:00",
10 "end": "2022-04-23T00:00:00",
11 "props": {
12 "fluentNetworkRef": "BASE_67"
13 }
14 },
15 "jobStart": "2022-09-24T02:00:00",
16 "jobsEnd": "2022-09-24T02:05:00",
17 "status": "SUCCESSFUL"
18 }
19}Structural Data Parameter Definitions
| Property | Description |
`customDateRange` | Optional parameter block injecting single-use timing limits. Removed upon successful run completion. |
`customDateRange.start` | Explicit bounding start timestamp for custom tracking loops. |
`customDateRange.end` | Explicit bounding end timestamp for custom tracking loops. |
`previousEndDate` | Persistence timestamp tracking engine history. Operates as baseline start metric for next processing loops. Defaults to Unix Epoch if absent. |
`props` | Custom structural parameter maps assigned to background handlers. |
`lastRun` | Audit container logging data variables, configurations, and state traces from the preceding run. |
`lastRun.param.start` | Start parameters utilized during previous cycle processing. |
`lastRun.param.end` | End parameters utilized during previous cycle processing. |
`lastRun.param.props` | Mapping variables utilized during previous cycle processing. |
`lastRun.jobStart` | System date-time log indicating when execution operations initiated. |
`lastRun.jobsEnd` | System date-time log indicating when execution operations concluded. |
`lastRun.status` | State tracking outcome evaluating process output: `FAILED` or `SUCCESSFUL`. |
Infrastructure Layout Frameworks
Build Properties Configuration Matrix
Route handlers interact with variable parameters specified directly within structural manifest parameters:1routes:
2 - name: "fc.connect.batch-inventory-sync"
3 handler: "BatchInventorySyncJob"
4 props:
5 page-size: 500General Infrastructure Configuration Hierarchy
1connector-name: commercetools
2configuration:
3 base-path-key: "fc.connect.<connector-name>"
4 account-mapping-key: "fc.connect.<connector-name>.account-mapping"
5 general-key: "fc.connect.<connector-name>.general"| Property | Description |
`connector-name` | Core formatting string used to build structural naming attributes and search path trees. Characters restricted to a-z and hyphens. |
`base-path-key` | Lookup reference parameter fetching all configuration maps matching an account-retailer context path. |
`account-mapping-key` | Configuration reference linking Fluent accounts with unique identifiers on external ecosystems. |
`general-key` | Fallback bucket parameters storing operational properties that do not match structural features. |
`log-level-key` | System property mapping to control framework log levels (DEBUG, INFO, WARN, ERROR). Applies on configuration refresh. |
Account Mapping Blueprint
1{
2 "fluent": {
3 "accountId": "FCTDEV",
4 "retailerId": "100"
5 },
6 "externalAccount": {
7 "projectKey": "my-project-qa",
8 "uid": "123"
9 }
10}General Catalog Setup Blueprint
1{
2 "productCatalogue": {
3 "entityRef": "PC:MASTER:100",
4 "entitySubType": "MASTER"
5 }
6}