Skip to main content
Loading

The routing section of aerospike-elasticsearch-outbound.yml

The routing section of the /etc/aerospike-elasticsearch-outbound/aerospike-elasticsearch-outbound.yml file controls how Aerospike records are routed to a destination index in Elasticsearch.

Routing Modeโ€‹

An outbound connector can route messages to one or more indexes. You can use the static routing mode to specify in advance the exact names of the indexes to route messages to. You can also use three different dynamic routing modes to have the outbound connector determine the names of indexes dynamically, based on information in records.

Staticโ€‹

  • static Always routes to the specified index.
  • skip Skip dispatch of record to Elasticsearch and ack success to XDR.

Dynamicโ€‹

  • namespace Uses the namespace of each Aerospike record as the name of the index to publish messages to.
  • set Uses the set of each Aerospike record as the name of the index to publish to. For records without a set, you can specify a fallback route that uses the "static" routing mode.
  • bin Uses the value of a bin in each Aerospike record as the name of the index to publish to. Only string, blob, and integer bin types are supported.

When you use one of the dynamic modes, you can transform the names of namespaces, sets, and bins by changing their case, removing whitespace, or using regular expressions to replace characters.

Static routingโ€‹

Here are the configuration options:

OptionRequiredExpected valueDescription
modeyesstaticSpecifies to write messages to the index that is named in the destination option.
destinationyesName of the destination index.

Exampleโ€‹

...
routing:
mode: static
destination: users
...

Skip routingโ€‹

For skip routing the configuration options are

OptionRequiredExpected valueDescription
modeyesskipSkip dispatch of record to Elasticsearch and ack success to XDR.

Exampleโ€‹

...
routing:
mode: skip
...

Namespace routingโ€‹

Here are the configuration options:

OptionRequiredExpected valueDescription
modeyesnamespaceSpecifies to write to index that have name that matches name of namespace in record metadata.
transformsnoList of transformations to apply to the namespace name. See the Transforming dynamically derived names section for details.

Exampleโ€‹

...
routing:
mode: namespace
transforms:
- trim
- regex:
pattern: '(.*):(.*)'
replacement: '$2:$1'
- regex:
pattern: '$'
replacement: ':please'
- uppercase
...

Set name routingโ€‹

Here are the configuration options:

OptionRequiredExpected valueDescription
modeyessetSpecifies to write to index that have name that matches name of a set in record metadata.
defaultnoDefault destination index to use in case the set name is missing in the record.
transformsnoList of transformations to apply to the set name. See the Transforming dynamically derived names section for details.

Exampleโ€‹

...
routing:
mode: set
transforms:
- trim
- regex:
pattern: '(.*):(.*)'
replacement: '$2:$1'
- regex:
pattern: '$'
replacement: ':please'
- uppercase
...

Bin value routingโ€‹

Here are the configuration options:

OptionRequiredExpected valueDescription
modeyesbinSpecifies to write to index that have name that matches value in the specified bin.
binyesName of the bin to pick value from.
defaultyesDefault destination index to use in case the bin is missing in the record.
transformsnoList of transformations to apply to the bin value. See the Transforming dynamically derived names section for details.

Exampleโ€‹

...
routing:
mode: bin
bin: category
transforms:
- trim
- regex:
pattern: '[^A-Za-z0-9]'
replacement: '-'
- lowercase
...

Custom Routingโ€‹

Record can also be routed with custom code. See Routing Transform.

Exampleโ€‹

...
routing:
mode: custom
class: com.aerospike.connect.outbound.example.GenerationRouter
...

Transforming dynamically derived namesโ€‹

You can configure a list of transforms that will be applied, in order, to the record's set name, namespace or bin value to derive the name of the destination index.

The following transformations are supported:

NameDescription
lowercaseConverts to lowercase.
uppercaseConverts to uppercase
trimTrims leading and trailing whitespace.
regexMatches against a regex pattern and replaces all occurrences with a replacement. The regex and replacement use Java regex syntax.

Exampleโ€‹

The following transform configuration trims the route, replaces all non-alphanumeric characters with '-', and then converts the result to lowercase.

...
routing:
mode: bin
bin: category
default: test-index
transforms:
- trim
- regex:
pattern: '[^A-Za-z0-9]'
replacement: '-'
- lowercase
...