How Metrics works
Author:
Kirill Gaiduk
Changed on:
9 July 2024
Overview
This document is intended to describe to implementers Metrics technical details within the Fluent Big Inventory product.
Pre-requisites:
- You should have knowledge of Metrics
- You should have knowledge of OMX Workflow Framework.
- You should have knowledge of Events.
- You should have knowledge of GraphQL.
Key points
- All customers' Metrics are isolated in the dedicated workspaces.
- Metrics are captured from the Fluent APIs, Orchestration Engine (Rubix), Batch Pre-Processing, and Inventory Feeds.
- Use and
`metricInstant`
GraphQL queries to retrieve the Metrics data.`metricRange`
Metrics Data Segregation
All customers' Metrics are isolated on the Fluent platform due to the Metrics Data Segregation.
The purpose of data segregation is to:
- Ensure data security, privacy, and compliance;
- Maintain the Metrics' data integrity;
- Prevent unauthorized access from different customers.
Metrics Managing Flow
Metrics managing flow includes the following steps:
1. We instrument the Application Source Code to capture the Platform Metrics as activity occurs.
Two primary applications play pivotal roles in the Inventory Processing flow:
- Fluent APIs act as an entry point, accepting external HTTP API requests from the customer.
The Metrics captured for Fluent APIs pertain to the requests that:- the Fluent platform has “received”,
- or the ones customers have “sent”.
- Orchestration engine (Rubix) processes the customer’s business logic by executing Orchestration event requests.
The Metrics captured here relate to the Fluent platform's requests “processed”.
2. The Metrics data is routed (through the Metrics Collector) to each customer-specific Metrics workspace by remote writing endpoint based on the
`account_id`
The Metrics data is stored in the Metrics workspace and ready for query.
Metrics Data Access Management
- Metrics API is introduced as part of GraphQL API.
- GraphQL API provides the authorization functionality:
- Validation of the user token correctness.
- Validation if the user has been granted the accessible roles/permissions.
- The introduction of GraphQL API as a proxy guarantees that all the Metrics queries requested by one user always belong to one specific customer/account rather than all customers' Metrics being accessible by any users.
Metrics API Querying Flow
Metrics API querying flow includes the following steps:
1. Two Metrics GraphQL queries are available for customers to retrieve the Metrics data:
`metricInstant`
`metricRange`
2. GraphQL API identifies the account ID for the user from:
- The user token,
- and request header.
`fluent.account`
Then Metrics API queries the corresponding Metrics workspace by the account ID.
3. After the Metrics API gets the Metrics workspace, it:
- Sends the PromQL (Prometheus Query Language) the user provides to the Metrics workspace.
- Receives the response from the Metrics workspace.
- Maps the response back to Metrics API schema (check the Metrics API Schema section in the Getting Started with Metrics API).
- Returns the Metrics query result to the user.