Ingestion API

Note

Version: 1.0.0
Base URL: https://next.itrslab.com/ingestion/api

The Ingestion API provides endpoints for managing data ingestion, filtering, throttling, and retention policies within the ITRS Analytics platform. This API allows you to control how data flows into the system and configure retention policies for different data types.

Authentication Copied

All endpoints require proper authentication. Include your authentication headers with each request.

Ingestion Filters Copied

Ingestion filters allow you to control which data points are ingested into the system. You can create dimension filters, expression filters, or Geneos filters to selectively include or exclude data based on various criteria.

Create Ingestion Filter Copied

POST /v1/create-ingestion-filter

Creates one of the ingestion filter types: dimension, expression, or Geneos.

Request Body Copied

Example Request (Dimension Filter) Copied

{
  "spec": {
    "dimensionFilter": {
      "dimensions": {
        "dimKey1": "dimValue1",
        "dimKey2": "dimValue2"
      },
      "dataPointTypes": ["DATAPOINT_TYPE_UNDEFINED"]
    }
  },
  "enabled": true,
  "name": "Example Dimension Filter"
}

Example Request (Expression Filter) Copied

{
  "spec": {
    "expressionFilter": {
      "expression": {
        "boolean": {
          "left": {
            "set": {
              "name": "container",
              "values": ["container1", "container2"]
            }
          },
          "right": {
            "comparison": {
              "name": "client",
              "value": "client1"
            }
          }
        }
      },
      "dataPointTypes": ["DATAPOINT_TYPE_UNDEFINED"]
    }
  },
  "enabled": true,
  "name": "Example Expression Filter"
}

Example Request (Geneos Filter) Copied

{
  "spec": {
    "geneosFilter": {
      "filters": [{
        "include": {
          "rows": ["rowFilter1", "rowFilter2"],
          "dataviews": ["dataviewFilter1", "dataviewFilter2"],
          "plugins": ["pluginFilter1", "pluginFilter2"],
          "gateways": ["gatewayFilter1", "gatewayFilter2"]
        },
        "exclude": {
          "rows": ["rowFilter1", "rowFilter2"],
          "dataviews": ["dataviewFilter1", "dataviewFilter2"],
          "plugins": ["pluginFilter1", "pluginFilter2"],
          "gateways": ["gatewayFilter1", "gatewayFilter2"]
        }
      }]
    }
  },
  "enabled": true,
  "name": "Example Geneos Filter"
}

Example Response Copied

{
  "filter": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "version": "1",
    "enabled": true,
    "spec": {
      "dimensionFilter": {
        "dimensions": {
          "dimKey1": "dimValue1",
          "dimKey2": "dimValue2"
        },
        "dataPointTypes": ["DATAPOINT_TYPE_UNDEFINED"]
      }
    },
    "name": "Example Dimension Filter"
  }
}

Get All Ingestion Filters Copied

GET /v1/ingestion-filters

Returns an array of all configured ingestion filters.

Parameters Copied

None required.

Example Response Copied

{
  "filters": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "version": "1",
      "enabled": true,
      "spec": {
        "dimensionFilter": {
          "dimensions": {
            "dimKey1": "dimValue1"
          },
          "dataPointTypes": ["DATAPOINT_TYPE_GAUGE"]
        }
      },
      "name": "Example Filter"
    }
  ]
}

Delete Ingestion Filter Copied

DELETE /v1/ingestion-filter/{id}

Deletes an existing ingestion filter identified by its unique ID.

Path Parameters Copied

Example Response Copied

{}

Modify Ingestion Filter Copied

POST /v1/modify-ingestion-filter

Modifies an existing ingestion filter. The supplied version number must match the currently stored version number.

Request Body Copied

Example Request Copied

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "version": "1",
  "spec": {
    "dimensionFilter": {
      "dimensions": {
        "dimKey1": "updatedValue1",
        "dimKey2": "updatedValue2"
      },
      "dataPointTypes": ["DATAPOINT_TYPE_GAUGE"]
    }
  },
  "enabled": true,
  "name": "Updated Filter Name"
}

Example Response Copied

{
  "version": "2"
}

Toggle Ingestion Filter Copied

POST /v1/toggle-ingestion-filter

Toggles the enabled field of an existing ingestion filter. The supplied version number must match the currently stored version number.

Request Body Copied

Example Request Copied

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "version": "1",
  "enabled": false
}

Example Response Copied

{
  "version": "2"
}

Ingestion Throttles Copied

Ingestion throttles allow you to control the rate at which data is ingested into the system to prevent overwhelming the platform.

Create Ingestion Throttle Copied

POST /v1/create-ingestion-throttle

Creates an ingestion throttle to limit data ingestion rates.

Request Body Copied

Example Request Copied

{
  "enabled": true,
  "spec": {
    "scope": "INGESTION_THROTTLE_SCOPE_METRICS",
    "maxRate": 1000,
    "burstFactor": 2
  }
}

Example Response Copied

{
  "throttle": {
    "id": "550e8400-e29b-41d4-a716-446655440001",
    "version": "1",
    "enabled": true,
    "spec": {
      "scope": "INGESTION_THROTTLE_SCOPE_METRICS",
      "maxRate": 1000,
      "burstFactor": 2
    }
  }
}

Get All Ingestion Throttles Copied

GET /v1/ingestion-throttles

Returns an array of all configured ingestion throttles.

Parameters Copied

None required.

Example Response Copied

{
  "throttles": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440001",
      "version": "1",
      "enabled": true,
      "spec": {
        "scope": "INGESTION_THROTTLE_SCOPE_METRICS",
        "maxRate": 1000,
        "burstFactor": 2
      }
    }
  ]
}

Delete Ingestion Throttle Copied

DELETE /v1/ingestion-throttle/{id}

Deletes an existing ingestion throttle identified by its unique ID.

Path Parameters Copied

Example Response Copied

{}

Modify Ingestion Throttle Copied

POST /v1/modify-ingestion-throttle

Modifies an existing ingestion throttle. The supplied version number must match the currently stored version number.

Request Body Copied

Example Request Copied

{
  "id": "550e8400-e29b-41d4-a716-446655440001",
  "version": "1",
  "enabled": true,
  "spec": {
    "scope": "INGESTION_THROTTLE_SCOPE_METRICS",
    "maxRate": 2000,
    "burstFactor": 3
  }
}

Example Response Copied

{
  "version": "2"
}

Toggle Ingestion Throttle Copied

POST /v1/toggle-ingestion-throttle

Toggles the enabled field of an existing ingestion throttle. The supplied version number must match the currently stored version number.

Request Body Copied

Example Request Copied

{
  "id": "550e8400-e29b-41d4-a716-446655440001",
  "version": "1",
  "enabled": false
}

Example Response Copied

{
  "version": "2"
}

Data Retention Copied

Data retention policies control how long different types of data are stored in the system before being automatically deleted.

Create Metric Retention Policy Copied

POST /v1/metric-retention-policy

Creates a metric retention policy with a valid metric name regex pattern, time bucket size, priority, and duration.

Request Body Copied

Example Request Copied

{
  "policy": {
    "pattern": "name_pattern_*",
    "bucket": "METRIC_BUCKET_RAW",
    "priority": 1,
    "duration": "604800s",
    "version": "1"
  }
}

Example Response Copied

{
  "version": "1"
}

Get All Retention Policies Copied

GET /v1/retention-policy

Returns two arrays of retention policies: metric and non-metric.

Parameters Copied

None required.

Example Response Copied

{
  "metricRetentionPolicies": [
    {
      "pattern": "name_pattern_*",
      "bucket": "METRIC_BUCKET_RAW",
      "priority": 1,
      "duration": "604800s",
      "version": "1"
    }
  ],
  "retentionPolicies": [
    {
      "name": "spans",
      "duration": "1209600s",
      "version": "1"
    }
  ]
}

Delete Metric Retention Policy Copied

POST /v1/delete-metric-retention-policy

Deletes an existing metric retention policy matching on regex pattern, time bucket size, and priority. The supplied version number must match the currently stored version number.

Request Body Copied

Example Request Copied

{
  "policy": {
    "pattern": "name_pattern_*",
    "bucket": "METRIC_BUCKET_RAW",
    "priority": 1,
    "version": "2"
  }
}

Example Response Copied

{
  "version": "3"
}

Update Metric Retention Policy Copied

POST /v1/update-metric-retention-policy

Updates an existing metric retention policy matching on regex pattern, time bucket size, and priority. The supplied version number must match the currently stored version number.

Request Body Copied

Example Request Copied

{
  "policy": {
    "pattern": "name_pattern_*",
    "bucket": "METRIC_BUCKET_RAW",
    "priority": 1,
    "duration": "1209600s",
    "version": "1"
  }
}

Example Response Copied

{
  "version": "2"
}

Update Non-Metric Retention Policy Copied

POST /v1/update-retention-policy

Updates a default non-metric retention policy matching on name. The duration is in seconds (between 60 mins and 10 years). The supplied version number must match the currently stored version number.

Request Body Copied

Example Request Copied

{
  "policy": {
    "name": "spans",
    "duration": "1209600s",
    "version": "1"
  }
}

Example Response Copied

{
  "version": "2"
}

Error Responses Copied

All endpoints may return error responses in the following format:

{
  "title": "Bad Request",
  "message": "The file name must be set",
  "details": {
    "type": "validation_error",
    "path": ".file.name"
  }
}

Common HTTP status codes:

• 200 OK - Request successful
• 400 Bad Request - Invalid request parameters
• 401 Unauthorized - Authentication required
• 404 Not Found - Resource not found
• 409 Conflict - Version mismatch or resource conflict
• 500 Internal Server Error - Server error

Schemas Copied

This section describes the data models used throughout the Ingestion API, organized alphabetically by schema name.

Any Copied

Generic container for arbitrary data with YAML serialization support.

CreateIngestionFilterRequestApp Copied

Create ingestion filter request schema.

CreateIngestionFilterResponse Copied

Response schema for creating an ingestion filter.

CreateIngestionThrottleRequest Copied

Create ingestion throttle request schema.

CreateIngestionThrottleResponse Copied

Response schema for creating an ingestion throttle.

CreateMetricRetentionPolicyRequest Copied

Create metric retention policy request schema.

CreateMetricRetentionPolicyResponse Copied

Response schema for creating a metric retention policy.

DeleteIngestionFilterResponse Copied

Response schema for deleting an ingestion filter (empty object).

DeleteIngestionThrottleResponse Copied

Response schema for deleting an ingestion throttle (empty object).

DeleteMetricRetentionPolicyRequest Copied

Delete metric retention policy request schema.

DeleteMetricRetentionPolicyResponse Copied

Response schema for deleting a metric retention policy.

DimensionFilter Copied

Filters data points based on dimension key-value pairs.

Supported Data Point Types:

• DATAPOINT_TYPE_UNDEFINED
• DATAPOINT_TYPE_COUNTER
• DATAPOINT_TYPE_DOWNSAMPLED_METRIC
• DATAPOINT_TYPE_ENTITY_ATTRIBUTE
• DATAPOINT_TYPE_ENTITY_GROUP_SNAPSHOT
• DATAPOINT_TYPE_GAUGE
• DATAPOINT_TYPE_GENERIC_EVENT
• DATAPOINT_TYPE_GENERIC_HISTOGRAM
• DATAPOINT_TYPE_LOG_EVENT
• DATAPOINT_TYPE_STATUS_METRIC
• DATAPOINT_TYPE_SIGNAL_EVENT
• DATAPOINT_TYPE_SNOOZE_EVENT
• DATAPOINT_TYPE_ENTITY_ATTRIBUTE_GROUP_SNAPSHOT
• DATAPOINT_TYPE_OPEN_TELEMETRY_SPAN_GROUP
• DATAPOINT_TYPE_OPEN_TELEMETRY_SPAN
• DATAPOINT_TYPE_OPEN_TELEMETRY_LOG
• DATAPOINT_TYPE_OPEN_TELEMETRY_METRIC
• DATAPOINT_TYPE_AUDIT_EVENT

Expression Copied

Complex expression object used in expression filters for advanced dimension matching. The structure supports boolean operations, set operations, and comparisons.

ExpressionFilter Copied

Filters data points using complex expression logic for dimension matching.

FilterSpec Copied

Defines the specification for different types of ingestion filters. Only one filter type should be specified.

GeneosDataFilter Copied

Individual Geneos filter with include/exclude selectors.

GeneosDataFilterSet Copied

Container for Geneos-specific filters that process dataview row data points.

Filter Application Logic:

• Filters are evaluated in the specified order
• Within each filter, exclude selectors take precedence over include selectors
• If no filters select a data point, it is excluded by default

GeneosDataSelector Copied

Selects Geneos data points based on path elements.

GetIngestionFiltersResponse Copied

Response schema for retrieving all ingestion filters.

GetIngestionThrottlesResponse Copied

Response schema for retrieving all ingestion throttles.

GetRetentionPoliciesResponse Copied

Response schema for retrieving all retention policies.

GoogleProtobufAny Copied

Protocol Buffers Any type for type-safe serialization.

IngestionFilter Copied

Represents a configured ingestion filter that controls which data points are ingested.

IngestionThrottle Copied

Represents a configured ingestion throttle that controls data ingestion rates.

MetricRetentionPolicy Copied

Represents a retention policy for metric data with regex pattern matching.

ModifyIngestionFilterRequestApp Copied

Modify ingestion filter request schema.

ModifyIngestionFilterResponse Copied

Response schema for modifying an ingestion filter.

ModifyIngestionThrottleRequest Copied

Modify ingestion throttle request schema.

ModifyIngestionThrottleResponse Copied

Response schema for modifying an ingestion throttle.

NamedAny Copied

Represents maps of Any as ordered (name,value) pairs.

RetentionPolicy Copied

Represents a retention policy for non-metric data types.

ThrottleSpec Copied

Defines the rate limiting configuration for ingestion throttles.

Supported Scopes:

• INGESTION_THROTTLE_SCOPE_UNDEFINED
• INGESTION_THROTTLE_SCOPE_METRICS

ToggleIngestionFilterRequest Copied

Toggle ingestion filter request schema.

ToggleIngestionFilterResponse Copied

Response schema for toggling an ingestion filter.

ToggleIngestionThrottleRequest Copied

Toggle ingestion throttle request schema.

ToggleIngestionThrottleResponse Copied

Response schema for toggling an ingestion throttle.

UpdateMetricRetentionPolicyRequest Copied

Update metric retention policy request schema.

UpdateMetricRetentionPolicyResponse Copied

Response schema for updating a metric retention policy.

UpdateRetentionPolicyRequest Copied

Update retention policy request schema.

UpdateRetentionPolicyResponse Copied

Response schema for updating a retention policy.