Skip to main content
Loading

Configure the Elasticsearch Client for Aerospike Connect for Elasticsearch

The es-client section of the aerospike-elasticsearch-outbound.yaml file specifies the configuration parameters for creating Elasticsearch Java Client. This client is used for all communication from Aerospike Connect for Elasticsearch to the Elasticsearch cluster.

Cluster configurationโ€‹

The cluster-config section defines the Elasticsearch cluster type.

OptionRequiredDescriptionDetails
typeYesThe type of the Elasticsearch cluster.The supported options are on-prem and cloud.

Each type has a specific required configuration.

TypeOptionDescriptionDetails
on-premnodesA list of Elasticsearch nodes to connect to. All valid properties of Node can be configured.Choose this option if you manage the Elasticsearch cluster, and you have access to the cluster nodes.
cloudcloud-idA valid elastic cloud cloudId that will route to a cluster. The cloudId is located in your Elastic Cloud user consoleChoose this option if you are using the Elastic Cloud hosted service.

Default Headersโ€‹

An optional default-headers section is a set of key-value pairs that sets the default request headers which are sent with each request.

Exampleโ€‹

...
es-client:
...
default-headers:
environment: prod
...
...

Auth configurationโ€‹

The auth-config section sets options for authenticating with the Elasticsearch cluster.

OptionRequiredDefaultDescriptionDetails
typeYesnoneThe type of authentication method.The supported options are basic, token, api-key, and none.

Some of these types have extra configuration options.

Config TypeOptionRequiredDescription
basicusernameYesUsername for basic auth.
basicpassword-fileYesFile path of the file containing the password for basic authentication.
tokentoken-fileYesFile path of the file containing the authentication token.
api-keyapi-key-id-fileYesFile path of the file containing the API key ID.
api-keyapi-key-secret-fileYesFile path of the file containing the API key Secret.

Transport optionsโ€‹

The transport-options section is optional. It sets configurations for all application-specific or request-specific options, including headers and query parameters.

OptionDefaultDescription
headersempty mapAll request specific headers.
parametersempty mapAll request specific headers query parameters.

Request Configโ€‹

The elasticsearch-request-config section sets options for customizing the underlying HTTP client at request level. RequestConfig has more details on some of these options:

OptionDefaultDescription
expect-continue-enabledfalseDetermines whether the 'Expect: 100-Continue' handshake is enabled for entity enclosing methods. Refer to RequestConfig for more details.
proxynullHTTP proxy for request execution. All valid properties of HttpHost can be configured.
local-addressnullLocal address for request execution.
stale-connection-check-enabledfalseWhether or not to use stale connection check.
cookie-specnullName of the cookie specification for HTTP state management.
redirects-enabledtrueWhether or not redirects are handled automatically.
relative-redirects-allowedtrueWhether or not relative redirects are rejected. HTTP specification requires the location value be an absolute URI.
circular-redirects-allowedfalseWhether or not circular redirects (redirects to the same location) are allowed.
max-redirects50Maximum number of redirects to be followed. The limit on number of redirects is intended to prevent infinite loops.
authentication-enabledtrueWhether or not authentication is handled automatically.
target-preferred-auth-schemesnullOrder of preference for supported authentication schemes in the form of a list when authenticating with the target host.
proxy-preferred-auth-schemesnullOrder of preference for supported authentication schemes in the form of a list when authenticating with the proxy host.
connection-request-timeout-1Timeout in milliseconds used when requesting a connection from the connection manager.
connect-timeout-1Timeout in milliseconds until a connection is established.
socket-timeout-1Socket timeout SO_TIMEOUT in milliseconds. The timeout for waiting for data, specifically, the maximum period of inactivity between two consecutive data packets.
content-compression-enabledtrueWhether or not the target server is asked to compress content.
normalize-uritrueWhether or not the client should normalize URIs in requests.

Other configurationsโ€‹

You can also configure a few optional parameters:

OptionDefaultDescription
enable-compressionfalseWhether the REST client should compress requests using gzip content encoding and add the "Accept-Encoding: gzip" header to receive compressed responses.
path-prefixEmpty StringSets the path's prefix for every request used by the HTTP client.
meta-header-enabledfalseWhether to send a X-Elastic-Client-Meta header that describes the runtime environment.
strict-deprecation-mode-enabledfalseWhether the REST client should return any response containing at least one warning header as a failure.
tls-confignullAn optional TLS configuration for connecting to the Elasticsearch cluster. The configuration options here are the same as Service section's TLS config.
max-conn-total0Maximum total connections to the Elasticsearch cluster.
max-conn-per-route0Maximum connection per route to the Elasticsearch cluster.
conn-time-to-live-millis-1Maximum time to live in milliseconds for persistent connections.
io-thread-countNumber of available processorsThe number of I/O dispatch threads to be used by the I/O reactor.
user-agentDefaults to aerospike-elasticsearch-outbound:{connector-version} where connector-version is the version of the deployed aerospike-elasticsearch-outbound connector.The User-agent to be set with every request.
cookie-management-disabledfalseDisable state (cookie) management.
auth-caching-disabledfalseDisable authentication scheme caching.
connection-state-disabledfalseDisable connection state tracking.

Exampleโ€‹

...
es-client:
cluster-config:
type: cloud
cloud-id: abc123
default-headers:
env: dev
enable-compression: true
path-prefix: /my/prefix
meta-header-enabled: true
strict-deprecation-mode-enabled: true
auth-config:
type: basic
username: elastic
password-file: /etc/aerospike-elasticsearch-outbound/secret/basic-auth-password.txt
tls-config:
trust-store:
store-file: /etc/aerospike-elasticsearch-outbound/secret/tls/ca.my-domain.com.truststore.jks
store-password-file: /etc/aerospike-elasticsearch-outbound/secret/tls/storepass
transport-options:
headers:
over-tls: true
parameters:
my-param1: hello-world
elasticsearch-request-config:
expect-continue-enabled: true
proxy:
hostname: localhost
port: 1234
scheme: http
local-address: 192.168.123.234
stale-connection-check-enabled: true
cookie-spec: test-cookie-spec
redirects-enabled: false
relative-redirects-allowed: false
circular-redirects-allowed: false
max-redirects: 5
authentication-enabled: true
target-preferred-auth-schemes:
- Basic
- NTLM
proxy-preferred-auth-schemes:
- CredSSP
connection-request-timeout: 2500
connect-timeout: 1000
socket-timeout: 10000
content-compression-enabled: false
normalize-uri: false
max-conn-total: 10
max-conn-per-route: 20
conn-time-to-live-millis: 30000
io-thread-count: 30
user-agent: my-test-webapp
cookie-management-disabled: false
auth-caching-disabled: true
connection-state-disabled: true
...