Skip to main content
Loading

Prometheus Integration

Prometheus integration is enabled by default for all HTTP source (outbound) connectors, and is enabled for source connectors compatible with Aerospike 5.0 and later when the manage subsection in the service section is configured in the connector's configuration file. The REST endpoint to integrate with Prometheus is manage/rest/v1/prometheus.

note

For an HTTP source connector, the service port and manage port are the same.

Prometheus Configuration for an Aerospike Connectorโ€‹

The scrape_configs section of Prometheus configuration has to point to the Aerospike Connector's manage REST API endpoint.

...
scrape_configs:
- job_name: aerospike-connect
metrics_path: /manage/rest/v1/prometheus
static_configs:
targets: # List of Manage Endpoints of all Aerospike Connectors exposing Prometheus metrics.
- outbound-connector-1:8081
- outbound-connector-2:8081
...

Aerospike Connector's Metrics available in Prometheusโ€‹

info

Connector exposes several metrics. We have only highlighted some of the important ones here.

Countersโ€‹

A counter is a cumulative metric that represents a single monotonically increasing counter whose value can only increase or be reset to zero on restart.

NameLabelDescriptionFrom Connector version
aerospike_kafka_outbound_connections{status=active}Count of active XDR connections to the outbound connector.4.0.0
aerospike_kafka_outbound_connections{status=closed}Count of closed XDR connections to the outbound connector.4.0.0
aerospike_kafka_outbound_requests{status=error}Count of requests that could not be written to the external Kafka system.3.0.0
aerospike_kafka_outbound_requests{status=parse-error}Count of XDR records which could not be parsed.3.0.0
aerospike_kafka_outbound_requests{status=skipped}Count of XDR records which were skipped and not written to the external Kafka system.3.0.0
aerospike_kafka_outbound_requests{status=success}Count of records written to the external Kafka system.3.0.0
aerospike_kafka_outbound_requests{status=queued}Current count of records being executed by the outbound connector.4.0.0
aerospike_kafka_outbound_kafka_producers{status=active}Count of active Kafka producers.4.0.0
aerospike_kafka_outbound_kafka_producers{status=closed}Count of closed Kafka producers.4.0.0

Gaugeโ€‹

A gauge always returns the current value of the metric and can increase or decrease.

NameDescriptionFrom Connector version
aerospike_kafka_outbound_requests_lagThe lag between update of a record in Aerospike and it reaching the Connector measured in milliseconds. Calculated as aerospike_record.lut - current_time. Its sampled every second based on the XDR record being processed at that time, it is approximate and not exact.4.0.0

Meterโ€‹

Meters measure the rate of events in the connector. It provides the total count of the metric, mean value and one-minute, five-minute, and fifteen-minute moving average values.

NameDescriptionFrom Connector version
aerospike_kafka_outbound_ordering_errorCount of records rejected to maintain an ordering guarantee of records having the same Aerospike key. XDR retries such a records.5.0.0

Histogramsโ€‹

Histograms calculate the distribution of a value. It provides the total count of the metric, maximum value, minimum value, mean value, and values at various quantiles (99%, 95%, etc).

NameDescriptionFrom Connector version
aerospike_kafka_outbound_record_payload_sizeSize of records received by the connector in Bytes.4.0.0

Timersโ€‹

Timers time the processing of tasks. A timer metric aggregates timing durations and provides duration statistics, plus throughput statistics. It provides the total count of the metric, maximum time, minimum time, mean time, times at various quantiles (99%, 95%, etc), and mean throughput value and one-minute, five-minute, and fifteen-minute moving average throughput values.

NameDescriptionFrom Connector version
aerospike_kafka_outbound_record_ack_queueTime spent by a processed XDR record in the ack-queue waiting to be acked to Aerospike XDR source.4.0.0
aerospike_kafka_outbound_record_dispatchTime spent dispatching an XDR record to the external Kafka system.4.0.0
aerospike_kafka_outbound_record_parsingTime spent parsing an XDR record.4.0.0
aerospike_kafka_outbound_requests_totalTotal time a XDR record has spent in the connector.3.0.0

Grafana Integrationโ€‹

Grafana dashboards can be integrated with Prometheus.

Grafana dashboards are packaged as JSON and can be imported.

The file path for the Aerospike Kafka source (outbound) connector is:

/opt/aerospike-kafka-outbound/grafana/aerospike-kafka-outbound-instances.json, /opt/aerospike-kafka-outbound/grafana/aerospike-kafka-outbound-summary.json     

A screenshot of the Grafana dashboard querying Prometheus for Aerospike Outbound metrics data.