Skip to main content
Loading

Metrics

Prometheusโ€‹

Configurationโ€‹

To configure a Prometheus metrics endpoint, you need to configure the HTTP endpoint in the Secret Agent's configuration file as below.

Sample Secret Agent's configuration file to configure the HTTP endpoint.

service:
http:
endpoint: 0.0.0.0:8080

Secret Agent uses the configured HTTP service to expose Prometheus endpoint using /manage/rest/v1/prometheus. Full URL will be something like http://localhost:8080/manage/rest/v1/prometheus

Exposed Metricsโ€‹

Secret Agent exposes metrics for

  1. Listeners like TCP, UDS.
  2. Secret Managers like AWS, GCP.

All metrics have the label module which value represents the module emitting the metric.

Metrics for Listenersโ€‹

Metrics NameDescriptionModulesType
aerospike_sa_connections_activeNumber of active connectionsTCP, UDSGauge
aerospike_sa_connections_openNumber of opened connectionsTCP, UDSCounter
aerospike_sa_connections_closedNumber of closed connectionsTCP, UDSCounter
aerospike_sa_connections_errNumber of connection errorsTCP, UDSCounter
aerospike_sa_read_errNumber of errors while reading from connectionsTCP, UDSCounter
aerospike_sa_write_errNumber of errors while writing to connectionsTCP, UDSCounter
aerospike_sa_tls_errNumber of errors during the tls handshake for connectionsTCP, UDSCounter
aerospike_sa_parse_errNumber of errors while parsing the requestsTCP, UDSCounter

Metrics for Secret Managersโ€‹

Metrics NameDescriptionModulesType
aerospike_sa_successNumber of successful fetch requestsAWS, GCPCounter
aerospike_sa_fetch_latency_p50p50 latency for fetch requestsAWS, GCPGauge
aerospike_sa_fetch_latency_p95p95 latency for fetch requestsAWS, GCPGauge
aerospike_sa_resource_errNumber of errors due to bad resource name in the requestAWS, GCPCounter
aerospike_sa_fetch_errNumber of errors while fetching secrets from external secret managersAWS, GCPCounter
aerospike_sa_key_not_found_errNumber of errors where the requested key is not found in the fetched key-value pairsAWSCounter
aerospike_sa_session_errNumber of errors while creating a client session to external secret managersAWS, GCPCounter
aerospike_sa_role_errNumber of errors while assuming role (AWS)/impersonating (GCP)AWS, GCPCounter

Custom labelsโ€‹

You can configure custom labels for Prometheus metrics as well. Labels can be provided as key-value pair in the metrics context.

Sample Secret Agent's configuration file to configure custom labels.

...
...
http:
endpoint: 0.0.0.0:8080
metrics:
prometheus:
labels:
label1: val1
label2: val2
...
...

Log Tickerโ€‹

Along with Prometheus metrics, Secret Agent also prints metrics in the log file every 10 seconds. Log ticker format is:

<module name>: <metric1> <val1>, <metric2> <val2>, ...

Sample ticker log.

[INFO] 2023/09/09 11:22:34 metrics.go:65: tcp: connections_active 1, connections_opened 2, connections_closed 1, connections_err 0, read_err 0, write_err 0, tls_err 0, parse_err 0
[INFO] 2023/09/09 11:22:34 metrics.go:65: uds: connections_active 0, connections_opened 0, connections_closed 0, connections_err 0, read_err 0, write_err 0, tls_err 0, parse_err 0
[INFO] 2023/09/09 11:22:34 metrics.go:65: aws: success 2, fetch_latency_p50 38, fetch_latency_p95 108, resource_err 0, fetch_err 0, key_not_found_err 0, session_err 0, role_err 0
[INFO] 2023/09/09 11:22:44 metrics.go:65: tcp: connections_active 1, connections_opened 2, connections_closed 1, connections_err 0, read_err 0, write_err 0, tls_err 0, parse_err 0
[INFO] 2023/09/09 11:22:44 metrics.go:65: uds: connections_active 0, connections_opened 0, connections_closed 0, connections_err 0, read_err 0, write_err 0, tls_err 0, parse_err 0
[INFO] 2023/09/09 11:22:44 metrics.go:65: aws: success 2, fetch_latency_p50 38, fetch_latency_p95 108, resource_err 0, fetch_err 0, key_not_found_err 0, session_err 0, role_err 0