Fluent Commerce Logo
Docs

commercetools Connector Extend Configuration

Essential knowledge

Intended Audience:

Technical User

Author:

Girish Padmanabha

Changed on:

17 June 2026

Overview

The commercetools Connector features a highly adaptable runtime architecture governed by four baseline configuration files: `application-connector.yml`, `application.yml`, `bootstrap.yml`, and `logback-spring.xml`. Together, these files define core connector settings, server/container properties, and logging rules.To manage multi-environment rollouts efficiently, you can override common baseline settings by creating dedicated files that follow standard Spring Boot profile conventions (e.g., `application-dev.yml`) to isolate your development, staging, and production environments.

Key points

  • What You Will Learn: You will understand how to safely configure, secure, and scale the bidirectional data flow between commercetools and Fluent Commerce across multi-profile cloud environments without modifying core connector code.
  • Core File Architecture: The connector requires a strict baseline of four configuration files (`application-connector.yml`, `application.yml`, `bootstrap.yml`, and `logback-spring.xml`). You can dynamically override any environment setting by appending standard Spring Boot profile suffixes (e.g., `application-dev.yml`).
  • Critical Operational Guardrails: * Job Serialization: The asynchronous job scheduler enforces strict concurrency control—multiple instances of the same data synchronization job cannot run simultaneously.
    • Data Retention: Manual `customDateRange` overrides are destructive parameters; the connector executes the targeted historical timeframe exactly once and automatically deletes the custom range to prevent looping data.
  • Security and Performance Impact: Moving or updating sensitive external platform endpoints requires configuring the connector's dedicated credential storage layer. Additionally, message processing throughout during peak traffic events is directly governed by adjusting the `pool-size` and queue listener properties.

Core Manifest Hierarchy

The connector runtime checks and parses a minimum layout of four configuration files during boot operations. Environment-specific overrides follow standard SpringBoot multi-profile layout conventions (e.g., appending the profile indicator to create target configurations like `application-dev.yml`).
  • `application-connector.yml`: Manages project and connector-specific route architectures and operational definitions.
  • `application.yml`: Declares core defaults, system variables, and global initialization baselines.
  • `bootstrap.yml`: Governs container layouts and configuration parameters matching specific server deployment contexts.
  • `logback-spring.xml`: Organizes logging appenders, metrics collection boundaries, and environmental log formatting rules.

Security & Secrets Blueprint

To connect securely to external engines, credential sets must reside within a verified, restricted vault location. Core service customized bindings can be registered via the SDK Customization Manual, while configuration practices follow the standard Credential / Secret Setup Guide

Ingress Listener Typologies

Global pipeline orchestration depends directly on top-level infrastructure setting arrays. External webhooks run independently of these configuration criteria.

Core Settings Config Matrix

PropertyRequiredDescription
`message-listener-mode`YesDefines the listener layout assigned exclusively to the connector message queue.
`batch-listener-mode`YesSets the ingestion layer properties used for the structural connector batch queue.
`available-listeners`YesEnumerates the complete array of target tracking listeners recognized by the running service instance.
1fluent-connect:
2  message-listener-mode: sqs
3  batch-listener-mode: sqs
4  available-listeners:
5    - "sqs"
6    - "kafka"

Individual Listener Allocation Blocks

Instances support any number of parallel processing threads running simultaneously. Each definition registers unique properties matching the structure below:
1listeners:
2    # listener configurion block 
3    listener-id:
4      name: "ENV_VARIABLE"
5      pool-size: 20
6      prefix: "my-prefix"
7      shutdown-timeout: 300
8      visibility-timeout: 120
9      poll-wait: 20
10      dlq-suffix: "-dlq"
11      retry: 5
12      type: sqs   
PropertyRequiredDefault ValueDescription
`listener-id`Yesn/aStructural identifier of the listener matching codes configured via `@ListenerInfo`.
`name`YesEmpty StringEnvironmental variable mapping containing the physical target queue identifier.
`type`YesNOT_DEFINEDStructural queuing type selected from supported platform listeners.
`fluent-account`Yesn/aDirecting target account reference used to isolate account execution.
`pool-size`No10Thread count parameters governing parallel message processing capacity.
`prefix`NodefaultFormatting tag assigned to execution threads for traceability.
`shutdown-timeout`No300Seconds allowed for active tasks to process prior to container termination commands.
`visibility-timeout`No120Invisibility window in seconds ensuring unique delivery while a node executes a task.
`poll-wait`No10Timing metrics in seconds indicating how long a listener waits to pull items from queues.
`dlq-suffix`No-dlqFormatting suffix appended to establish Dead Letter Queue designations.
`retry`No2Count defining queue retry limits prior to rerouting messages into the target DLQ.
`max-allowed-pull-size`No10Upper bounds limiting maximum message count per queue pull request.
`props`Non/aExtensible map layout storing custom structural parameters.

Engine Chronological Orchestration (Job Scheduler)

The application delegates tracking timelines to external workflow management networks. Tasks trigger asynchronously via structured REST invocations, instantly processing execution parameters before providing standard `200 OK` return codes.

Job Ingestion Rest Endpoints 

1PUT https://<domain>/api/v1/fluent-connect/scheduler/add/<fluent-account>/<fluent-retailer>/<job-name>
  • `fluent-account`: Fluent infrastructure account reference.
  • `fluent-retailer`: Isolated context identifier tracking retailer context.
  • `job-name`: Assigned handler class targeting the batch operation.
1PUT https://localhost:8080/api/v1/fluent-connect/scheduler/add/cnctdev/1/batch-inventory-sync

Pre- and Post-Execution Lifecycle Rules

Every job executing inside the connector environment complies uniformly with the following structural rules:
  • Concurrency blocks ensure duplicate instances cannot run simultaneously.
  • Context models extract runtime configurations directly out of active Fluent settings.
  • Logical modules compute bounded date ranges to isolate target execution deltas.
  • Core handler logic runs the assigned integration process in UTC time.
  • Post-execution routines persist final runtime parameters for subsequent calculations.

Configuration Management Classes

System jobs balance parameters between two discrete layers:
  • Build Profiles: Implemented directly as local codebase variables within deployable manifests (e.g., properties maps inside YAML routing lists).
  • Runtime Settings: Managed dynamically as state properties objects stored inside Fluent settings. Keys compile structurally using prefix concatenation parameters:
    YAML
    job-scheduler:`
    `
     config-prefix:` `"fc.connect.<connector-name>.data-sync"`
    `
Note: A job named `batch-inventory-sync` maps to configuration key `fc.connect.<connector-name>.data-sync.batch-inventory-sync`.

Delta Tracking Chronological Calculus

Every engine loop receives computed timestamp bounds via its job context, allowing delta-mode updates. If `previousEndDate` parameter inputs are missing or unassigned, start markers default to Unix Epoch time.
  • Scenario A (First Run, No Baseline Date): From: Epoch time $\rightarrow$ To: Now.
  • Scenario B (First Run, Baseline Date Present): From: Explicit Date $\rightarrow$ To: Now.
  • Scenario C (Subsequent Cycles): From: Preceding End DateTime (`previousEndDate`) $\rightarrow$ To: Now.

Configuration Profiles & State Traces

Baseline Job Profile Setup (Unexecuted State)

1{
2    "previousEndDate": "",
3    "props": {
4        "fluentNetworkRef": "BASE_67"
5    }
6}

First Execution State Footprint

1{
2  "previousEndDate": "2022-09-22T03:00:00",
3  "props": {
4    "fluentNetworkRef": "BASE_67"
5  },
6  "lastRun": {
7    "param": {
8      "start": "1970-01-01T00:00:00",
9      "end": "2022-09-22T03:00:00",
10      "props": {
11        "fluentNetworkRef": "BASE_67"
12      }
13    },
14    "jobStart": "2022-09-22T03:00:00",
15    "jobsEnd": "2022-09-22T03:05:00",
16    "status": "SUCCESSFUL"
17  }
18}

Subsequent Execution State Footprint

1{
2  "previousEndDate": "2022-09-23T02:00:00",
3  "props": {
4    "fluentNetworkRef": "BASE_67"
5  },
6  "lastRun": {
7    "param": {
8      "start": "2022-09-22T03:00:00",
9      "end": "2022-09-23T02:00:00",
10      "props": {
11        "fluentNetworkRef": "BASE_67"
12      }
13    },
14    "jobStart": "2022-09-23T02:00:00",
15    "jobsEnd": "2022-09-23T02:05:00",
16    "status": "SUCCESSFUL"
17  }
18}

User-Defined Overrides State Footprint

Users can manually inject single-use chronological constraints. The connector applies these properties for one run cycle before removing the object properties.
1{
2  "customDateRange": {
3    "start": "2022-04-22T00:00:00",
4    "end": "2022-04-23T00:00:00"
5  },
6  "previousEndDate": "2022-09-23T02:00:00",
7  "props": {
8    "fluentNetworkRef": "BASE_67"
9  },
10  "lastRun": {
11    "param": {
12      "start": "2022-09-22T03:00:00",
13      "end": "2022-09-23T02:00:00",
14      "props": {
15        "fluentNetworkRef": "BASE_67"
16      }
17    },
18    "jobStart": "2022-09-23T02:00:00",
19    "jobsEnd": "2022-09-23T02:05:00",
20    "status": "SUCCESSFUL"
21  }
22}

Automatic Resume State Footprint (Post-Override Cleanup)

1{
2  "previousEndDate": "2022-09-23T02:00:00",
3  "props": {
4    "fluentNetworkRef": "BASE_67"
5  },
6  "lastRun": {
7    "param": {
8      "start": "2022-04-22T00:00:00",
9      "end": "2022-04-23T00:00:00",
10      "props": {
11        "fluentNetworkRef": "BASE_67"
12      }
13    },
14    "jobStart": "2022-09-24T02:00:00",
15    "jobsEnd": "2022-09-24T02:05:00",
16    "status": "SUCCESSFUL"
17  }
18}

In-Flight Lock Tracking Footprint

1{
2  "previousEndDate": "2022-09-23T02:00:00",
3  "executionStart" : "2022-09-24T02:00:00",
4  "props": {
5    "fluentNetworkRef": "BASE_67"
6  },
7  "lastRun": {
8    "param": {
9      "start": "2022-04-22T00:00:00",
10      "end": "2022-04-23T00:00:00",
11      "props": {
12        "fluentNetworkRef": "BASE_67"
13      }
14    },
15    "jobStart": "2022-09-24T02:00:00",
16    "jobsEnd": "2022-09-24T02:05:00",
17    "status": "SUCCESSFUL"
18  }
19}

Structural Data Parameter Definitions

PropertyDescription
`customDateRange`Optional parameter block injecting single-use timing limits. Removed upon successful run completion.
`customDateRange.start`Explicit bounding start timestamp for custom tracking loops.
`customDateRange.end`Explicit bounding end timestamp for custom tracking loops.
`previousEndDate`Persistence timestamp tracking engine history. Operates as baseline start metric for next processing loops. Defaults to Unix Epoch if absent.
`props`Custom structural parameter maps assigned to background handlers.
`lastRun`Audit container logging data variables, configurations, and state traces from the preceding run.
`lastRun.param.start`Start parameters utilized during previous cycle processing.
`lastRun.param.end`End parameters utilized during previous cycle processing.
`lastRun.param.props`Mapping variables utilized during previous cycle processing.
`lastRun.jobStart`System date-time log indicating when execution operations initiated.
`lastRun.jobsEnd`System date-time log indicating when execution operations concluded.
`lastRun.status`State tracking outcome evaluating process output: `FAILED` or `SUCCESSFUL`.

Infrastructure Layout Frameworks

Build Properties Configuration Matrix

Route handlers interact with variable parameters specified directly within structural manifest parameters:
1routes:
2    - name: "fc.connect.batch-inventory-sync"
3      handler: "BatchInventorySyncJob"
4      props:
5        page-size: 500

General Infrastructure Configuration Hierarchy

1connector-name: commercetools
2configuration:
3    base-path-key: "fc.connect.<connector-name>"
4    account-mapping-key: "fc.connect.<connector-name>.account-mapping"
5    general-key: "fc.connect.<connector-name>.general"
PropertyDescription
`connector-name`Core formatting string used to build structural naming attributes and search path trees. Characters restricted to a-z and hyphens.
`base-path-key`Lookup reference parameter fetching all configuration maps matching an account-retailer context path.
`account-mapping-key`Configuration reference linking Fluent accounts with unique identifiers on external ecosystems.
`general-key`Fallback bucket parameters storing operational properties that do not match structural features.
`log-level-key`System property mapping to control framework log levels (DEBUG, INFO, WARN, ERROR). Applies on configuration refresh.

Account Mapping Blueprint

1{
2    "fluent": {
3        "accountId": "FCTDEV",
4        "retailerId": "100"
5    },
6    "externalAccount": {
7        "projectKey": "my-project-qa",
8        "uid": "123"
9    }
10}

General Catalog Setup Blueprint

1{
2    "productCatalogue": {
3        "entityRef": "PC:MASTER:100",
4        "entitySubType": "MASTER"
5    }
6}