Skip to main content
Loading

Flat JSON Format for Aerospike Kafka Source (Outbound) Connector

This page specifies the Flat JSON data serialization format used by the Aerospike Kafka Source (Outbound) Connector. It describes how Aerospike notification messages are represented as Flat JSON objects. Each message represents an event that occurred in an Aerospike database cluster. Different kinds of messages are used to describe events such as record creation/update and record deletions.

Key Format

The record key has the following properties

FieldTypeDescription
namespacestringNamespace of the Aerospike record.
setstringSet of the Aerospike record. May be missing in some cases.
userKeynumber, bytes (Base64 encoded) or stringThe user key of the Aerospike record. Is available if user key is stored on the Aerospike server.
digestbytes (Base64 encoded)Digest of the Aerospike record.

Example Key

{
"namespace": "users",
"set": "premium",
"userKey": "id123",
"digest": "IBlTW5m3UGqbFxfrsoDxCrLkKYQ="
}

Message Format

Each message is a JSON object with the metadata included in the specified metadata-key in the config and all the bins present as top level objects.

Message metadata

The message metadata properties are:

MetadataTypeDescription
msgstringWrite/Delete operation.
namespacestringNamespace of the Aerospike record.
setstringSet of the Aerospike record.
userKeylong, double, bytes or stringUser key of the Aerospike record. Present only if it is a write operation and the user key is stored on the Aerospike server.
digestbytesDigest of the Aerospike record.
genintGeneration of the Aerospike record.
lutlongTime when the record was last updated, in milliseconds since the Unix epoch. It is available whenever the Aerospike server ships last-update time. [1][2]
expintTime when the record will expire, in seconds since the Unix epoch. Zero means the record will not expire. Present only in write operation.
durablebooleanWhether the delete is durable. Present only in delete operation.
note

All metadata is affected by both delete and write operations, except where the description indicates otherwise.

[1] When the Aerospike server does not ship lut, Aerospike Kafka source (outbound) connector versions earlier than 4.0.0 ship lut as zero.

[2] Breaking Change When the Aerospike server ships lut, Aerospike Kafka source (outbound) connector versions earlier than 4.0.0 ship lut as a value of the data type "integer".

Example Write Message

Record with bins color and size.

{
"metadata": {
"msg": "write",
"namespace": "users",
"set": "premium",
"userKey": 7612,
"gen": 4,
"lut": 1617167159548,
"digest": "IBlTW5m3UGqbFxfrsoDxCrLkKYQ=",
"exp": 1682797792
},
"color": "red",
"size": 123
}

Example Delete Message

{
"metadata": {
"msg": "delete",
"namespace": "users",
"digest": "IBlTW5m3UGqbFxfrsoDxCrLkKYQ=",
"gen": 4,
"lut": 1617167159548,
"durable": false
}
}