Authors:
Kirill Gaiduk, Yulia Andreyanova
Changed on:
19 June 2024
This document is intended to describe to implementers which Metrics are available within the Fluent Big Inventory product and what kind of data they provide to achieve Platform Observability.
Pre-requisites:
Metrics are numerical measurements that are:
An is typically used to log a specific activity of what has happened within the system.
While the Metrics (or Metrics API) themselves do not capture events, they provide a capability to reflect an aggregated view for a range of captured events, which include:
These Metrics are associated with the following endpoints:
The following Metric types are currently available:
Labels are saved as a part of the specific Metric to enable Metrics data filtering when fetched with the Metrics API queries. Label examples include (a list of all Metrics and supported labels can be found in the Available Metrics section below):
`le`
stands for "less than or equal to". It labels the buckets in a histogram and represents the upper bound of each bucket.The following Metrics are currently available:
Name | Type | Labels | Description |
core_event_received_total | Counter | account_id, retailer_id, event_name, entity_type, source | The number of events received by the Fluent platform. |
core_event_last_received_seconds | Gauge | account_id, retailer_id, event_name, entity_type, source | The last timestamp of events received by the Fluent platform. |
Name | Type | Labels | Description |
rubix_event_received_total | Counter | account_id, retailer_id, event_name, entity_type, source | The number of events received by the engine () from the queue or direct HTTP API. |
rubix_event_inflight_latency_seconds | Histogram | account_id, retailer_id, event_name, entity_type, source | The amount of time an stays in the internal queue before receives it. |
rubix_event_inflight_latency_seconds_sum | Counter | account_id, retailer_id, event_name, entity_type, source | Total accumulated time of events in the internal queue before receives them. |
rubix_event_inflight_latency_seconds_count | Counter | account_id, retailer_id, event_name, entity_type, source | The number of observed events in the internal queue before receives them. |
rubix_event_inflight_latency_seconds_bucket | Counter | account_id, retailer_id, event_name, entity_type, source, le | Bucketed counts of latencies in the internal queue. |
rubix_event_runtime_seconds | Histogram | account_id, retailer_id, event_name, entity_type, source, status | The amount of time takes to execute an . |
rubix_event_runtime_seconds_sum | Counter | account_id, retailer_id, event_name, entity_type, source, status | Total accumulated time taken by to execute events. |
rubix_event_runtime_seconds_count | Counter | account_id, retailer_id, event_name, entity_type, source, status | The number of observed events executed. |
rubix_event_runtime_seconds_bucket | Counter | account_id, retailer_id, event_name, entity_type, source, status, le | Bucketed counts of runtimes by . |
The Metrics are captured from the Inventory Deduplication job:
Name | Type | Labels | Description |
bpp_records_processed_total | Counter | account_id, run_id, stage, first_batch_received, deduplication_finished | The total amount of all batch items processed by the Deduplication job. The The |
bpp_records_unchanged_total | Counter | account_id, run_id, stage | The total amount of batch items that were filtered out by the Deduplication job. |
bpp_records_changed_total | Counter | account_id, run_id, stage | The total amount of batch items that were considered “changed” and sent to by the Deduplication job. |
bpp_last_run_timestamp_seconds | Gauge | account_id, run_id, stage, status | The timestamp for when the given Batch Pre-Processing operation has been completed. Status: |
The Metrics are captured from the Inventory Data Loading job:
Name | Type | Labels | Description |
feed_sent_total | Counter | account_id, feed_ref, run_id, data_type | The total number of records exported by the Data Loading job, split by data type. Data type: |
feed_last_run_timestamp_seconds | Gauge | account_id, feed_ref, run_id, status | The timestamp for when the Data Loading job completed. Status: |
The Metrics capturing is a foundation and a starting point for building a reliable data source to enable Platform Observability. The following example for Update end-to-end flow (via the customer’s request based on (POS) data) is intended to serve as a reference for Metrics Capturing .
The correspondent Metrics (the number of events received and their runtime) are logged upon: