How Inventory Batch Pre-Processing works

Get Started
Functionality
Learning
Extension
Dev Tooling
APIs
Releases

Essential knowledge

Intended Audience:

Technical User

Authors:

Girish Padmanabha, Kirill Gaiduk

Changed on:

23 Mar 2026

Overview

This document outlines the functionality of Inventory Batch Pre-Processing, offering insights into how unchanged records are filtered during the inventory ingestion process using Inventory Batches.

Key points

Prerequisites: You should have knowledge of Global Inventory, Inventory Batch Pre-Processing, and Workflow Engine
Efficient Processing: Inventory Batch Pre-Processing handles batch inventory updates before they reach the inventory workflows, streamlining the process
Change Detection: Identifying inventory records that have changed, ensuring the workflow engine processes only necessary updates
Performance Improvement: Reduces workflow engine load by filtering unchanged records, enabling faster and more efficient inventory processing
Enhanced Flexibility: Supports additional fields in batch records, allowing for more precise control and management of inventory updates

Architecture

In traditional inventory update processes, batch updates are split into individual records and added directly to the Inventory Queue for processing. This can be inefficient, especially when many records haven't changed and don't require processing.Batch Pre-Processing introduces a more efficient flow:

Initial Analysis: Instead of immediately processing each record, the system first analyzes the batch to identify which records have actually changed
Selective Processing: The workflow engine forwards only the records identified as changed to the Inventory Queue for processing
Optimized Workflow: This reduces unnecessary workload on the workflow engine, improving overall system performance and reducing processing time

Components

The Batch Pre-Processing system consists of two main components:

1. Loader Job

Purpose: The Loader Job loads existing inventory data from the database. This includes current quantities, statuses, and any active transient inventory quantities.Function:

Data Gathering: Collects the current state of inventory positions and quantities
Preparation for Comparison: Prepares data for the Comparison Job to use in determining changes

2. Comparison Job

Purpose: The Comparison Job compares the new batch updates against the existing inventory data loaded by the Loader Job.Function:

Change Detection: Determines whether each inventory update represents a change that needs processing
Criteria for Change:
- Inventory Quantity Identification
- Quantity and Status Comparison
- Transient Inventory Quantities Check

Result: Only updates marked as changed are forwarded to the Inventory Queue for processing.

Understanding Transient Inventory Quantities

Definition: Transient inventory quantities are temporary adjustments due to sales, reservations, corrections, or other activities
Significance: Their presence indicates recent activity affecting the inventory position, necessitating reprocessing of the update

Decision Logic Flow

Deduplication Logic Overview

During preprocessing, the first step is to filter and route duplicate records across all batches in the current execution run. We achieve this by creating a composite key for each record, consisting of `event`, `catalogueRef`, and `ref` from the inventory items in the batch. If any of these fields are missing, default values are applied:

ref: If not provided, it is derived using the pattern `<skuRef>:<locationRef>:<catalogueType>:<type>`.
event: Defaults to `"InventoryChanged"` if not specified.
catalogueRef: Defaults to `"DEFAULT:<retailerId>"` when missing.
catalogueType: Derived from the type of `catalogueRef`.
type: Defaults to `"LAST_ON_HAND"` if not provided.

Once the composite keys are established, duplicates are identified by matching these keys across all batches. For each set of duplicates:

Records from the latest batch (based on sent time) are prioritized and used for processing.
If duplicates exist within the same batch, the selection of a duplicate for further processing is arbitrary.

This ensures that the most recent information from the latest batch is processed while older duplicates are discarded. This approach maintains data consistency and accuracy throughout the workflow.

Here's a detailed breakdown of how the Comparison Job decides whether an update is "changed" or "unchanged" for each inventory record in the batches processed for the current preprocessing execution:

Attribute-Only Updates

Batch Pre-Processing evaluates changes only based on Inventory Quantity identity, quantity, status, and transient conditions.Updates that modify other attributes (such as `expectedOn`, `expiresOn`, `condition`, or segmentation fields) do not affect change detection and may be filtered out.To reflect such changes, create a new Inventory Quantity record instead of modifying the existing one.

How Inventory Batch Pre-Processing works

Overview

Key points

Architecture

Components

1. Loader Job

2. Comparison Job

Decision Logic Flow

Related content

Inventory Batch Pre-Processing

Enable Batch Inventory Pre-Processing

Inventory Batch Pre-Processing: Technical Guide with Examples

Inventory Batch Events

Girish Padmanabha

Contributors:

Building Blocks

By Type

Helpful Resources

Quick Links