Inventory Batch Pre-Processing: Technical Guide with Examples
Author:
Fluent Commerce
Changed on:
11 Feb 2025
Overview
This page describes how Batch Pre-Processing (BPP) works and what technical aspects need to be considered as part of an implementation.
Key points
- Optimize Inventory Updates: Batch Pre-Processing enhances system efficiency by processing only necessary changes, reducing unnecessary load on the workflow engine.
- Structured Batch Updates: Properly structuring your batch updates with the correct use of fields like
`ref`
,`type`
, and`status`
is essential for accurate change detection and effective use of Batch Pre-Processing. - Essential Knowledge: Even if you don't delve into all the details, remember that precise batch update structures are crucial. Pay special attention to how the
`ref`
field is constructed and the default behaviors when certain fields are not specified. - New Fields and Defaults: Be aware of the new fields introduced by this feature and understand the default transient types and statuses. This knowledge will help you implement Batch Pre-Processing correctly and avoid potential pitfalls.
Pre-Requisites:
Batch Pre-Processing optimizes the update by analyzing batch updates before they reach the engine. This ensures that only necessary changes are processed, improving efficiency and performance.
This guide aims to help you understand:
- How Batch Pre-Processing works.
- How to structure your batch updates.
- How different scenarios are handled.
- The default behaviors when certain fields are not specified.
Change Detection Criteria
The following criteria to determine if an update represents a change:
- Inventory Quantity Identification
- Quantity and Status Comparison
- Transient Inventory Quantities Check
1. Inventory Quantity Identification
Determines how each quantity (IQ) is identified for comparison.
Aspect | Description | Default Behavior | Notes |
Type Handling | - Default Type: If |
| Ensures consistent identification when |
Reference ( | - Optional: When provided, | Constructed using | Facilitates precise IQ identification and de-duplication. |
Example of Reference (`ref`
) Construction:
`ref = <skuRef>:<locationRef>:<catalogueType>:<type>`
Components
Component | Description | Default |
| Product reference | Provided in the update |
| reference | Provided in the update |
| Catalogue type |
|
| IQ type |
|
Example
Given:
`skuRef`
:`SKU12345`
`locationRef`
:`LOC67890`
`catalogueRef`
:`CAT54321`
(type:`"BACK_ORDER"`
)`type`
: Not provided
Sample Constructed `ref`
:
`SKU12345:LOC67890:BACK_ORDER:LAST_ON_HAND`
2. Quantity and Status Comparison
Compares the updated quantities and statuses against existing data.
Comparison Type | Description | Outcome if Different |
Quantity ( | Compares the | The update is marked as changed. |
Status ( | If | The update is marked as changed. |
Notes:
- If the
`status`
field is not provided in the update, the status comparison is skipped.
3. Transient Inventory Quantities Check
Ensures that temporary quantities are factored in to determine change criteria
Aspect | Description | Outcome if Condition Met |
Transient Types | Includes temporary types such as | |
Matching Transients | Checks for any matching transient IQs of designated types and statuses associated with the position. Do note for performance reasons, any condition with the status "INACTIVE" is ignored | If matching transients exist, the update is marked as changed. |
Notes:
- Matching transients indicate ongoing temporary changes that may affect the accuracy of the update.
Summary of Change Detection Process
- Identify IQ:
- Use the provided
`ref`
or construct it based on`skuRef`
,`locationRef`
,`catalogueType`
, and`type`
.
- Use the provided
- Compare Quantities and Statuses:
- Check if
`qty`
differs. - If
`status`
is provided, check if it differs.
- Check if
- Check for Matching Transients:
- Determine if there are matching transient IQs that could affect the update.
- Mark as Changed:
- If any of the above comparisons indicate a difference, mark the update as changed.
Result
Only updates that are marked as changed based on the above criteria are forwarded to the Inventory Queue for further processing. This ensures that only relevant and necessary updates impact the system, maintaining data integrity and operational efficiency.
Below are examples of batch payloads and how the system processes them.
Example 1 - Update with Provided `ref`
and `type`
(PURCHASE_ORDER)
Batch Payload:
1{
2 "action": "UPSERT",
3 "entityType": "INVENTORY",
4 "entities": [
5 {
6 "ref": "SKU-001:LOC-001:DEFAULT:PURCHASE_ORDER",
7 "type": "PURCHASE_ORDER",
8 "skuRef": "SKU-001",
9 "locationRef": "LOC-001",
10 "qty": 100,
11 "retailerId": 1
12 }
13 ]
14}
Explanation:
- Ref Provided:
`"SKU-001:LOC-001:DEFAULT:PURCHASE_ORDER"`
- Process:
- Find IQ with provided
`ref`
. - Quantity Comparison:
- If the existing
`qty`
≠`100`
, the update is changed.
- If the existing
- Transient Check:
- Uses default transient types and statuses.
- If No IQ Found:
- The update is changed and processed as a new IQ.
- Find IQ with provided
Example 2 - Update with `status`
Change and Provided `ref`
and `type`
Batch Payload:
1{
2 "action": "UPSERT",
3 "entityType": "INVENTORY",
4 "entities": [
5 {
6 "ref": "SKU-002:LOC-002:DEFAULT:PURCHASE_ORDER",
7 "type": "PURCHASE_ORDER",
8 "status": "INACTIVE",
9 "skuRef": "SKU-002",
10 "locationRef": "LOC-002",
11 "qty": 50,
12 "retailerId": 1
13 }
14 ]
15}
Explanation:
- Ref Provided:
`"SKU-002:LOC-002:DEFAULT:PURCHASE_ORDER"`
- Status Provided:
`"INACTIVE"`
- Process:
- Find IQ with provided
`ref`
. - Quantity and Status Comparison:
- If the existing qty ≠ 50 OR the existing status ≠ "INACTIVE", the update is changed.
- Transient Check:
- Uses default transient types and statuses.
- If No IQ Found:
- The update is changed and processed as a new IQ.
- Find IQ with provided
Example 3 - Update with Custom `catalogueRef`
and Custom Transient Types
Batch Payload:
1{
2 "action": "UPSERT",
3 "entityType": "INVENTORY",
4 "catalogueRef": "CATALOG-001",
5 "conditions": {
6 "hasRelatedInventoryQuantities": [
7 {"type": "RESERVATION", "status": "ACTIVE"},
8 {"type": "RETURN", "status": "CREATED"}
9 ]
10 },
11 "entities": [
12 {
13 "ref": "SKU-003:LOC-003:BACK_ORDER:LAST_ON_HAND",
14 "type": "LAST_ON_HAND",
15 "status": "INACTIVE",
16 "skuRef": "SKU-003",
17 "locationRef": "LOC-003",
18 "qty": 200,
19 "retailerId": 1
20 }
21 ]
22}
Explanation:
- CatalogueRef Provided:
`"CATALOG-001"`
- Catalogue Type: Suppose
`"BACK_ORDER"`
- Ref Provided:
`"SKU-003:LOC-003:BACK_ORDER:LAST_ON_HAND"`
- Conditions Provided:
- Transient types:
`"RESERVATION"`
and`"RETURN"`
- Statuses:
`"ACTIVE"`
and`"CREATED"`
- Transient types:
- Process:
- Find IQ with the provided
`ref`
in`CATALOG-001`
. - Quantity and Status Comparison.
- Transient Check:
- Uses provided transient types and statuses.
- Find IQ with the provided
Example 4 - Update with Custom `catalogueRef`
, Provided `ref`
, and Custom Transient Types
Batch Payload:
1{
2 "action": "UPSERT",
3 "entityType": "INVENTORY",
4 "catalogueRef": "CATALOG-002",
5 "conditions": {
6 "hasRelatedInventoryQuantities": [
7 {"type": "ON_HOLD", "status": "ACTIVE"},
8 {"type": "ON_HOLD", "status": "CREATED"}
9 ]
10 },
11 "entities": [
12 {
13 "ref": "SKU-004:LOC-004:FUTURE:PURCHASE_ORDER",
14 "type": "PURCHASE_ORDER",
15 "skuRef": "SKU-004",
16 "locationRef": "LOC-004",
17 "qty": 75,
18 "retailerId": 1
19 }
20 ]
21}
Explanation:
- CatalogueRef Provided:
`"CATALOG-002"`
- Catalogue Type: Suppose
`"FUTURE"`
- Ref Provided:
`"SKU-004:LOC-004:FUTURE:PURCHASE_ORDER"`
- Process:
- Find IQ with the provided
`ref`
in`CATALOG-002`
. - Quantity Comparison.
- Transient Check:
- Uses provided transient types and statuses.
- Find IQ with the provided
Example 5 - Update Without `ref`
or `type`
, Using Custom Transient Types
Batch Payload:
1{
2 "action": "UPSERT",
3 "entityType": "INVENTORY",
4 "catalogueRef": "DEFAULT:1",
5 "conditions": {
6 "hasRelatedInventoryQuantities": [
7 {"type": "SALE", "status": "ACTIVE"},
8 {"type": "SALE", "status": "CREATED"},
9 {"type": "DELTA", "status": "ACTIVE"},
10 {"type": "DELTA", "status": "CREATED"}
11 ]
12 },
13 "entities": [
14 {
15 "skuRef": "SKU-005",
16 "locationRef": "LOC-005",
17 "qty": 60,
18 "retailerId": 1
19 }
20 ]
21}
Explanation:
- Type Defaulted to:
`"LAST_ON_HAND"`
- Catalogue Type:
`"DEFAULT"`
- Ref Constructed as:
`"SKU-005:LOC-005:DEFAULT:LAST_ON_HAND"`
- Process:
- Find IQ with constructed ref in the default catalog.
- Quantity Comparison.
- Transient Check:
- Uses provided transient types and statuses.
Example 6 - Basic Update Without `ref`
, `type`
, or Custom Transient Types
Batch Payload:
1{
2 "action": "UPSERT",
3 "entityType": "INVENTORY",
4 "entities": [
5 {
6 "skuRef": "SKU-006",
7 "locationRef": "LOC-006",
8 "qty": 90,
9 "retailerId": 1
10 }
11 ]
12}
Explanation:
- Type Defaulted to:
`"LAST_ON_HAND"`
- Catalogue Type:
`"DEFAULT"`
- Ref Constructed as:
`"SKU-006:LOC-006:DEFAULT:LAST_ON_HAND"`
- Process:
- Find IQ with constructed ref in the default catalog.
- Quantity Comparison.
- Transient Check:
- Uses default transient types and statuses.
Example 7 - Update with Provided `ref`
and Default `type`
(`LAST_ON_HAND`
)
Batch Payload:
1{
2 "action": "UPSERT",
3 "entityType": "INVENTORY",
4 "entities": [
5 {
6 "ref": "SKU-007:LOC-007:DEFAULT:LAST_ON_HAND",
7 "type": "LAST_ON_HAND",
8 "skuRef": "SKU-007",
9 "locationRef": "LOC-007",
10 "qty": 110,
11 "retailerId": 1
12 }
13 ]
14}
Explanation:
- Ref Provided:
`"SKU-007:LOC-007:DEFAULT:LAST_ON_HAND"`
- Process:
- Find IQ with the provided ref in the default catalogue.
- Quantity Comparison.
- Transient Check:
- Uses default transient types and statuses.
Example 8 - Update with Provided `ref`
Without Specifying `type`
Batch Payload:
1{
2 "action": "UPSERT",
3 "entityType": "INVENTORY",
4 "entities": [
5 {
6 "ref": "SKU-008:LOC-008:DEFAULT:LAST_ON_HAND",
7 "skuRef": "SKU-008",
8 "locationRef": "LOC-008",
9 "qty": 130,
10 "retailerId": 1
11 }
12 ]
13}
Explanation:
- Ref Provided:
`"SKU-008:LOC-008:DEFAULT:LAST_ON_HAND"`
- Process:
- Find IQ with the provided ref in the default catalogue.
- Quantity Comparison.
- Transient Check:
- Uses default transient types and statuses.
Example 9 - Update with Provided `ref`
, `type`
, and `status`
Batch Payload:
1{
2 "action": "UPSERT",
3 "entityType": "INVENTORY",
4 "entities": [
5 {
6 "ref": "SKU-009:LOC-009:DEFAULT:LAST_ON_HAND",
7 "status": "ACTIVE",
8 "type": "LAST_ON_HAND",
9 "skuRef": "SKU-009",
10 "locationRef": "LOC-009",
11 "qty": 150,
12 "retailerId": 1
13 }
14 ]
15}
Explanation:
- Ref Provided:
`"SKU-009:LOC-009:DEFAULT:LAST_ON_HAND"`
- Status Provided:
`"ACTIVE"`
- Process:
- Find IQ with provided
`ref`
. - Quantity and Status Comparison.
- Transient Check:
- Uses default transient types and statuses.
- Find IQ with provided
Example 10 - Update with Provided `type`
and `status`
, Without `ref`
Batch Payload:
1{
2 "action": "UPSERT",
3 "entityType": "INVENTORY",
4 "entities": [
5 {
6 "status": "ACTIVE",
7 "type": "LAST_ON_HAND",
8 "skuRef": "SKU-010",
9 "locationRef": "LOC-010",
10 "qty": 170,
11 "retailerId": 1
12 }
13 ]
14}
Explanation:
- Ref Constructed as:
`"SKU-010:LOC-010:DEFAULT:LAST_ON_HAND"`
- Status Provided:
`"ACTIVE"`
- Process:
- Find IQ with constructed
`ref`
. - Quantity and Status Comparison.
- Transient Check.
- Find IQ with constructed
Example 11 - Update with Provided `type`
(`PURCHASE_ORDER`
) and `status`
, Without `ref`
Batch Payload:
1{
2 "action": "UPSERT",
3 "entityType": "INVENTORY",
4 "entities": [
5 {
6 "status": "ACTIVE",
7 "type": "PURCHASE_ORDER",
8 "skuRef": "SKU-011",
9 "locationRef": "LOC-011",
10 "qty": 190,
11 "retailerId": 1
12 }
13 ]
14}
Explanation:
- Ref Constructed as:
`"SKU-011:LOC-011:DEFAULT:PURCHASE_ORDER"`
- Status Provided:
`"ACTIVE"`
- Type Provided:
`"PURCHASE_ORDER"`
- Process:
- Find IQ with constructed
`ref`
. - Quantity and Status Comparison.
- Transient Check.
- Find IQ with constructed
Best Practices
- Include
`ref`
When Possible:- Providing a unique
`ref`
ensures precise identification of the IQ and avoids unnecessary processing.
- Providing a unique
- Specify
`type`
and`status`
as Needed:- Explicitly specifying
`type`
and`status`
allows the system to target the correct IQ and reflect the intended changes.
- Explicitly specifying
- Use
`catalogueRef`
when there are multiple Catalogues:- Ensure updates are applied to the correct inventory catalogue by providing the appropriate
`catalogueRef`
. - Remember that the catalogue type is also used in the
`ref`
construction. - Ensure you have the workflow ready for different Inventory catalogue types
- Ensure updates are applied to the correct inventory catalogue by providing the appropriate
- Define Custom Transient Types and Statuses:
- Use
`conditions.hasRelatedInventoryQuantities`
to customize transient checks based on your business needs.
- Use
- Monitor Processing Using Recommended Tools:
- Utilize the Metrics API and orchestration audit events for real-time monitoring and error detection.
- Use Different Rulesets for Entry Points Based on IQ Types:
- Assign different rulesets as entry points for various IQ types. This enables precise control over behavior and the ruleset flow for each type if you need custom logic to handle IQ-specific input attributes
- Explicitly Bypass Transient Checks if Necessary:
- To bypass transient checks entirely, set:
`"conditions": {`
"hasRelatedInventoryQuantities": []
} - This ensures no default transient IQ types are applied.
- To bypass transient checks entirely, set:
- Include Relevant Fields:
- When submitting batch updates, include
`ref`
,`type`
,`status`
, and attributes if applicable. This ensures the system identifies the right record and processes changes accurately.
- When submitting batch updates, include
- Understand Default Behaviors:
- If
`type`
is not provided, it defaults to`"LAST_ON_HAND"`
. - If
`ref`
is not provided, the system constructs it based on`skuRef`
,`locationRef`
, and`type`
.
- If
Frequently Asked Questions (FAQ)
Q1: What happens if I don't provide the new optional fields in my batch updates?
A: The system will default to using standard identification methods based on `skuRef`
and `locationRef`
, with `type`
defaulting to `"LAST_ON_HAND"`
. It will proceed accordingly, using these defaults for comparison.
Q2: Can I update inventory quantities other than `"LAST_ON_HAND"`
?
A: Yes, by specifying the `type`
field in your batch update, you can target different types of quantities, such as `"PURCHASE_ORDER", "DELTA", "FUTURE" etc`
.
Q3: How does the system handle the `ref`
field in updates?
A: If you provide a reference, the system will use it to identify the specific quantity. If the `ref`
doesn't match any existing record, the update is considered changed and processed as a new record for the given catalogue.
Q4: What should I do if I need to monitor the processing of a specific batch?
A: Since batches are merged during processing, use the Metrics API and audit events to monitor individual updates and identify any issues.
Q5: How can I bypass transient checks in my updates?
A: To skip transient checks, explicitly set:
`"conditions": {`
"hasRelatedInventoryQuantities": []
}
This prevents the system from applying default transient IQ types during processing.
Q6: Why should I use different rulesets as entry points for IQ types?
A: Assigning different rulesets for various IQ types allows you to control the behavior and flow of rulesets effectively. This approach ensures that each IQ type is processed according to its unique requirements.
Q7: Are there any changes required on my end to start using Batch Pre-Processing?
A: Ensure that your batch updates are formatted correctly and include the new optional fields for better control. Also, adjust your monitoring practices to use the recommended tools.