Skip to main content
Loading
Version: Operator 2.5.0

Configuration Settings for an Aerospike Cluster on Kubernetes

Aerospike cluster configuration settings are in the Aerospike cluster Custom Resource (CR). You can edit this file to make changes to your Aerospike cluster.

The Operator Custom Resource Definition (CRD) specifies the CR that the Operator uses to manage the cluster.

Example CRโ€‹

This basic CR is included as an example to get you started. It creates a simple cluster with no storage, using data-in-memory (DIM).

For a more realistic real-world example, we recommend the example rack-enabled cluster CR.

These and other example CRs are stored in the main Aerospike Kubernetes Operator repository.

apiVersion: asdb.aerospike.com/v1beta1
kind: AerospikeCluster
metadata:
name: aerocluster
namespace: aerospike
spec:
size: 2
image: aerospike/aerospike-server-enterprise:6.1.0.1

podSpec:
multiPodPerHost: true

storage:
filesystemVolumePolicy:
cascadeDelete: true
initMethod: deleteFiles
volumes:
- name: workdir
source:
persistentVolume:
storageClass: ssd
volumeMode: Filesystem
size: 3Gi
aerospike:
path: /opt/aerospike
- name: aerospike-config-secret
source:
secret:
secretName: aerospike-secret
aerospike:
path: /etc/aerospike/secret

aerospikeAccessControl:
users:
- name: admin
secretName: auth-secret
roles:
- sys-admin
- user-admin

aerospikeConfig:
service:
feature-key-file: /etc/aerospike/secret/features.conf
security: {}
network:
service:
port: 3000
fabric:
port: 3001
heartbeat:
port: 3002
namespaces:
- name: test
memory-size: 3000000000
replication-factor: 2
storage-engine:
type: memory

Configurationโ€‹

The initial part of the CR selects the CRD and the namespace to use for the Aerospike cluster.

apiVersion: asdb.aerospike.com/v1beta1
kind: AerospikeCluster
metadata:
name: aerocluster
namespace: aerospike

Specโ€‹

The spec section defines the cluster's configurations.

FieldRequiredTypeDefaultDescription
size
Dynamic
YesIntegerThe size/number of Aerospike node pods to run for this cluster.
image
Dynamic Rolling restart
YesStringThe official Aerospike Enterprise Server docker image to use for the node in the cluster.
podSpec
Dynamic Rolling restart
NoStructureConfigures the Kubernetes pod running Aerospike server. See Pod Spec for details.
storage
Dynamic
NoStructureRequired for persistent namespaces and for Aerospike work directory, unless the validation policy skips validating persistence of the work directory. See Storage for details.
aerospikeNetworkPolicy
Dynamic Rolling restart
NoStructureConfigures IP and port types used for access and fabric. See Network Policy for details.
aerospikeAccessControl
Dynamic
NoStructureRequired if Aerospike security is enabled. See Access Control for details
aerospikeConfig
Dynamic Rolling restart
YesconfigMapA free form configMap confirming to the configuration schema for the deployed Aerospike server version. See Aerospike Config for details.
rackConfig
Dynamic
NoStructureConfigures the operator to deploy rack aware Aerospike cluster. Pods will be deployed in given racks based on given configuration. See Rack Config for details.
operatorClientCert
Dynamic
With TLSStructureConfigures the TLS certs used by the operator when connecting to the Aerospike cluster. See Operator Client Certs for details.
validationPolicy
Dynamic
NoStructureConfigures the custom resource validation. See Validation Policy for details.
seedsFinderServices
Dynamic
NoStructurecreates additional Kubernetes service that allow clients to discover Aerospike cluster nodes. See Seeds Finder Services for details.
rosterNodeBlockList
Dynamic
NoList of StringConfigures a list of nodeIDs which are blocked from roster in a Strong Consistency setup

|

Operator Client Certโ€‹

The operator needs to connect as an Aerospike client to perform management asinfo calls. If the Aerospike service is configured to use TLS for clients, you need to specify the certificates the client uses to make these calls.

FieldRequiredTypeDefaultDescription
tlsClientNameNoStringIf specified, this name will be added to tls-authenticate-client list by the operator so that it will always to allowed by the Aerospike servers.
secretCertSource*YesStructureThe operator certs should be read from a secret. See Secret Cert Source for details.
certPathInOperator*YesStructureThe operator certs should be read from files mounted into the operator container. See Cert Path In Operator Source for details.

* Exactly one for secretCertSource or certPathInOperator must be specified

Secret Cert Sourceโ€‹

Specifies that the certificate should be read from a Kubernetes Secret.

FieldRequiredTypeDefaultDescription
secretNameYesStringThe name of the secret.
secretNamespaceNoStringOperator's namespaceThe namespace containing the secret.
caCertsFilenameNoStringThe name of the secret key /file containing the CA certificate.
clientCertFilenameNoStringThe name of the secret key /file containing the operator's certificate.
clientKeyFilenameNoStringThe name of the secret key /file containing the operator's secret key.

Cert Path In Operator Sourceโ€‹

Specifies that the operator's certificate files are mounted onto the operator's containers.

FieldRequiredTypeDefaultDescription
caCertsPathNoStringThe path to the file containing the CA certificate.
clientCertPathNoStringThe path to the file containing the operator's certificate.
clientKeyPathNoStringThe path to the file containing the operator's secret key.

Pod Specโ€‹

Configures the Kubernetes pod running Aerospike server. Sidecar containers for monitoring or running connectors can be added to each Aerospike pod.

FieldRequiredTypeDefaultDescription
metadataNoStructureKubernetes Annotations and Labels to add to pods. See metadata for details
multiPodPerHostNoBooleanFalseIndicates if the operator should run multiple pods per Kubernetes cluster host.
affinityNoKubernetes Pod AffinityKubernetes Affinity rules for pod placement. These rules will be merged with affinity rules generated by the operator.
tolerationsNoKubernetes Pod TolerationsKubernetes Toleration for Aerospike pod placement
nodeSelectorNoMap from string to stringNode selector constraints for the Aerospike pods.
hostNetworkNoBooleanFalseEnables host networking for the pod. To enable hostNetwork, multiPodPerHost must be false.
dnsPolicyNoKubernetes DNSPolicyDnsPolicy to use. If hostNetwork is true and policy is not specified, it defaults to ClusterFirstWithHostNet
aerospikeContainerNoStructureConfigures the aerospike-server container created by operator. See Aerospike Container for details.
aerospikeInitContainerNoStructureConfigures the aerospike-init container created by operator. See Aerospike Init Container for details.
sidecarsNoList of Container structuresList of side containers to run along with the main Aerospike server container. Volume mounts are not supported. See storage for attaching volumes to sidecars.
initContainersNoList of Container structuresList of init containers before running Aerospike and the sidecar containers.
securityContextNoStructure PodSecurityContextSecurityContext holds pod-level security attributes and common container settings
imagePullSecretsNoList of TypedLocalObjectReferenceImagePullSecrets is an optional list of references to secrets in the same namespace to use for pulling any of the images used by this PodSpec.

See Monitoring for details on configuring monitoring sidecars or Aerospike containers.

Metadataโ€‹

Kubernetes annotations and labels to add to the pods.

FieldRequiredTypeDefaultDescription
annotationsNoMap from annotation name to its valueKubernetes Annotations
labelsNoMap from label name to its valueKubernetes Labels

Aerospike Containerโ€‹

Configures the aerospike-server container created by operator.

FieldRequiredTypeDefaultDescription
securityContextNoStructure SecurityContextKubernetes security context for the Aerospike container.
resourcesNoStructure ResourceRequirementsConfigures resources requirements and limits like CPU or memory for the Aerospike container.

Aerospike Init Containerโ€‹

Configures the aerospike-init container created by operator.

FieldRequiredTypeDefaultDescription
imageRegistryNoStringdocker.ioName of image registry for Aerospike init container image.
securityContextNoStructure SecurityContextKubernetes security context for the Aerospike init container.
resourcesNoStructure ResourceRequirementsConfigures resources requirements and limits like CPU or memory for the Aerospike init container.

Storageโ€‹

The storage section configures persistent volume devices to provision and attach to the containers.

This section is required by default for persisting the Aerospike work directory. Store the work directory on persistent storage to ensure pod restarts do not reset Aerospike server metadata files.

This section is also required if Aerospike namespaces require persistent storage.

FieldRequiredTypeDefaultDescription
filesystemVolumePolicy
Dynamic
NoStructureVolume policy for filesystem volumes
blockVolumePolicy
Dynamic
NoStructureVolume policy for block volumes
cleanupThreads
Dynamic
NoInteger1Defines the maximum number of disk cleanup threads(dd or blkdiscard) per init container that are used to wipe or init disk volumes.
Volumes
Dynamic
NoList of Volumes StructuresList of Volumes to attach to Aerospike pods. Persistent storage volumes cannot be added or removed dynamically.

Volume Policyโ€‹

Specifies persistent volume policy to determine how new volumes are initialized.

FieldRequiredTypeDefaultDescription
initMethod
Dynamic
NoEnumnoneControls how the volumes are initialized when the persistent volume is attached the first time to a pod. Valid values are none, dd, blkdiscard, deleteFiles
wipeMethod
Dynamic
NoEnumdd for block volumes and deleteFiles for filesystem volumesControls how the volumes are wiped for special upgrades/downgrades. Valid values are dd, blkdiscard, deleteFiles
cascadeDelete
Dynamic
NoBooleanfalseCascadeDelete determines if the persistent volumes are deleted after the pods these volumes binds to are terminated and removed from the cluster
note

blkdiscard only works for devices which support TRIM. For AWS please refer to the storage volumes guide to check TRIM support. If TRIM is not supported please use the slower dd to initialize/wipe your devices. For other devices please verify the support for TRIM command with your vendor.

InitMethodโ€‹

For filesystem volumes, initMethod can be none or deleteFiles. For block volumes, initMethod can be none, dd or blkdiscard.

WipeMethodโ€‹

For filesystem volumes, wipeMethod can be only deleteFiles. For block volumes, wipeMethod can be dd or blkdiscard.

caution

When using blkdiscard you need to be sure that the drive deterministically returns zeros after TRIM (RZAT). You should check with your hardware provider or cloud provider. If you are unsure, it is safer to wipe the device using dd as the wipe method.

Volumeโ€‹

Describes volumes to be created and attached to the init containers and main Aerospike container, as well as other sidecars.

FieldRequiredTypeDefaultDescription
nameYesStringA name identifying this volume, which must be unique among all created volumes.
sourceYesStructureSpecifies the source for this volume. See Volume Source for details
aerospikeNoStructureSpecifies how this volume is attached to the main Aerospike server container. See AerospikeServerVolumeAttachment for details.
sidecarsNoStructureAttaches this volume to other sidecar containers. See Volume Attachment for details.
initContainersNoStructureAttaches this volume to other init containers. See Volume Attachment for details.
initMethod
Dynamic
NoEnumnoneControls how this volume is initialized when the persistent volume is attached the first time to a pod. Valid values are none, dd, blkdiscard, deleteFiles
cascadeDelete
Dynamic
NoBooleanfalseCascadeDelete determines if the persistent volume is deleted after the pod this volume binds to is terminated and removed from the cluster

For filesystem volumes, initMethod can be none or deleteFiles. For block volumes, initMethod can be none, dd or blkdiscard.

note

blkdiscard only works for devices that support TRIM. For AWS please refer to the storage volumes guide to check TRIM support. If TRIM is not supported please use the slower dd to initialize your devices. For other devices please verify the support for TRIM command.

Aerospike Server Volume Attachmentโ€‹

Specifies attaching a volume to the main Aerospike server container.

FieldRequiredTypeDefaultDescription
pathYesStringThe path to attach the volume to the Aerospike Container.

Volume Attachmentโ€‹

Specifies attaching a volume to a sidecar container.

FieldRequiredTypeDefaultDescription
containerNameYesStringThe name of the container to attach this volume to.
pathYesStringThe path to attach the volume to the Aerospike Container.

Volume Sourceโ€‹

A Volume source specifies the source for an attached volume. Volumes are created using a volume source that can be one of the following:

  • Persistent volume
  • EmptyDir
  • Secret
  • ConfigMap

Persistent Volumeโ€‹

Specifies a persistent volume to claim and attach to Aerospike pods.

FieldRequiredTypeDefaultDescription
storageClassYesStringThe name of the storage class to use.
volumeModeYesEnum - Filesystem, BlockSpecified the mode this volume should be created with. Filesystem creates a pre-formatted filesystem. Block mode creates a raw device.
sizeYesString QuantityThe device size to be provisioned. E.g. 5Gi
accessModesNoList of PersistentVolumeAccessModeAllowed access modes to the volume. E.g. ReadWriteOnce, ReadOnlyMany, ReadWriteMany
selectorNoLabelSelectorOnly the volumes whose labels match the selector will be claimed.
initMethod
Dynamic
NoEnumnoneControls how this volume is initialized when the persistent volume is attached the first time to a pod. Valid values are none, dd, blkdiscard, deleteFiles
cascadeDelete
Dynamic
NoBooleanfalseCascadeDelete determines if the persistent volume is deleted after the pod this volume binds to is terminated and removed from the cluster

EmptyDirโ€‹

EmptyDir are volumes to be used as temporary working disk space. See the official documentation for options.

      - name: tempFiles
aerospike:
path: /opt/aerospike/temp
source:
emptyDir: {}

Secretโ€‹

A Kubernetes Secret can be mounted as a volume. You can use a Secret for mounting the Aerospike license file (features.conf) or sensitive files like TLS certificates and credentials onto the containers.

- name: aerospike-config-secret
source:
secret:
secretName: aerospike-secret
aerospike:
path: /etc/aerospike/secret

ConfigMapโ€‹

A Kubernetes ConfigMap can be mounted as a volume. ConfigMaps hold non-confidential data like configuration for applications running in your custom init containers or sidecars.

- name: app-config
source:
configMap:
# Provide the name of the ConfigMap you want to mount.
name: app-config
# An optional array of keys from the ConfigMap to create as files
items:
- key: "game.properties"
path: "game.properties"
- key: "user-interface.properties"
path: "user-interface.properties"

Validation Policyโ€‹

This section configures the policy for validating the cluster CR.

FieldRequiredTypeDefaultDescription
skipWorkDirValidate
Dynamic
NoBooleanfalseIf true skips validating that the Aerospike work directory is stored on a persistent volume.
skipXdrDlogFileValidate
Dynamic
NoBooleanfalseIf true skips validating that the XDR digest log is stored on a persistent volume.

Network Policyโ€‹

This section configures IP and port types used for access, alternate access, TLS access, TLS alternate access, fabric and TLS fabric endpoints on the Aerospike cluster.

Five types of endpoint configurations are supported.

  • pod uses the Kubernetes pod IP and Aerospike port that works to connect from other pods in the same Kubernetes cluster.
  • hostInternal uses the Kubernetes cluster node's host IP and a mapped Aerospike port that works to connect from the VPC or internal network used by the Kubernetes cluster.
  • hostExternal uses the Kubernetes cluster node's host external/public IP and a mapped Aerospike port that works to connect from the external network.
  • configuredIP uses the IP address configured by the user in the Kubernetes node labels, and a mapped Aerospike port, that is used to connect from the internal/external network. Label "aerospike.com/configured-access-address" in the Kubernetes node is used as accessAddress, and label "aerospike.com/configured-alternate-access-address" in the Kubernetes node is used as alternateAccessAddress.
  • customInterface uses the IP address assigned to the custom interfaces attached to the pod and Aerospike port, that is used to connect from the same custom interfaces network. These custom interfaces are given separately as a list of NetworkAttachmentDefinition CRs and also, must be defined in the pod annotation k8s.v1.cni.cncf.io/networks in order to assign network interfaces to the pod.
FieldRequiredTypeDefaultDescription
access
Dynamic Rolling restart
NoEnum [pod, hostInternal, hostExternal, configuredIP, customInterface]hostInternalConfigures Aerospike access endpoint.
customAccessNetworkNames
Dynamic Rolling restart
NoList of stringCustomAccessNetworkNames is the list of the pod's network interfaces used for Aerospike access address. Each element in the list is specified with a namespace and the name of a NetworkAttachmentDefinition, separated by a forward slash (/). These elements must be defined in the pod annotation k8s.v1.cni.cncf.io/networks in order to assign network interfaces to the pod. Required with 'customInterface' access type.
alternateAccess
Dynamic Rolling restart
NoEnum [pod, hostInternal, hostExternal, configuredIP, customInterface]hostExternalConfigures Aerospike alternate access endpoint.
customAlternateAccessNetworkNames
Dynamic Rolling restart
NoList of stringCustomAlternateAccessNetworkNames is the list of the pod's network interfaces used for Aerospike alternate access address. Each element in the list is specified with a namespace and the name of a NetworkAttachmentDefinition, separated by a forward slash (/). These elements must be defined in the pod annotation k8s.v1.cni.cncf.io/networks in order to assign network interfaces to the pod. Required with 'customInterface' alternateAccess type.
tlsAccess
Dynamic Rolling restart
NoEnum [pod, hostInternal, hostExternal, configuredIP, customInterface]hostInternalConfigures Aerospike TLS access endpoint.
customTLSAccessNetworkNames
Dynamic Rolling restart
NoList of stringCustomTLSAccessNetworkNames is the list of the pod's network interfaces used for Aerospike TLS access address. Each element in the list is specified with a namespace and the name of a NetworkAttachmentDefinition, separated by a forward slash (/). These elements must be defined in the pod annotation k8s.v1.cni.cncf.io/networks in order to assign network interfaces to the pod. Required with 'customInterface' tlsAccess type.
tlsAlternateAccess
Dynamic Rolling restart
NoEnum [pod, hostInternal, hostExternal, configuredIP, customInterface]hostExternalConfigures Aerospike TLS alternate endpoint.
customTLSAlternateAccessNetworkNames
Dynamic Rolling restart
NoList of stringCustomTLSAlternateAccessNetworkNames is the list of the pod's network interfaces used for Aerospike TLS alternate access address. Each element in the list is specified with a namespace and the name of a NetworkAttachmentDefinition, separated by a forward slash (/). These elements must be defined in the pod annotation k8s.v1.cni.cncf.io/networks in order to assign network interfaces to the pod. Required with 'customInterface' tlsAlternateAccess type.
fabricNoEnum [customInterface]Configures Aerospike fabric endpoint.
customFabricNetworkNamesNoList of stringCustomFabricNetworkNames is the list of the pod's network interfaces used for Aerospike fabric address. Each element in the list is specified with a namespace and the name of a NetworkAttachmentDefinition, separated by a forward slash (/). These elements must be defined in the pod annotation k8s.v1.cni.cncf.io/networks in order to assign network interfaces to the pod. Required with 'customInterface' fabric type.
tlsFabricNoEnum [customInterface]Configures Aerospike TLS fabric endpoint.
customTLSFabricNetworkNamesNoList of stringCustomTLSFabricNetworkNames is the list of the pod's network interfaces used for Aerospike TLS fabric address. Each element in the list is specified with a namespace and the name of a NetworkAttachmentDefinition, separated by a forward slash (/). These elements must be defined in the pod annotation k8s.v1.cni.cncf.io/networks in order to assign network interfaces to the pod.Required with 'customInterface' tlsFabric type.

Aerospike Access Controlโ€‹

Provides Aerospike access control configuration for the Aerospike cluster.

FieldRequiredTypeDefaultDescription
roles
Dynamic
NoList of StructuresA list of Role structures with an entry for each role.
users
Dynamic
NoList of StructuresA list of User structures with an entry for each user. Required if Aerospike security is enabled.

If the Aerospike cluster has security enabled an entry for the "admin" user having at least "sys-admin" and "user-admin" roles is mandatory.

Aerospike Roleโ€‹

Configures roles for the Aerospike cluster.

FieldRequiredTypeDefaultDescription
nameYesStringsThe name of this role.
privileges
Dynamic
YesList of StringsThe privileges to grant this role.
whitelist
Dynamic
NoList of StringsWhitelist of host address allowed for this role.
readQuota
Dynamic
NoIntegerspecifies permitted rate of read records for current role (the value is in RPS)
writeQuota
Dynamic
NoIntegerspecifies permitted rate of writes for current role (the value is in RPS)

Aerospike Userโ€‹

Configures users for the Aerospike cluster.

FieldRequiredTypeDefaultDescription
nameYesStringsThe name of this user.
secretName
Dynamic
YesStringThe name of the secret containing this user's password.
roles
Dynamic
YesList of StringsThe roles to grant to this user.

Aerospike Config Secretโ€‹

Configures the name of the secret to use and the mount path to mount the secret files on the container.

FieldRequiredTypeDefaultDescription
secretName
Dynamic
YesStringThe name of the secret
mountPath
Dynamic
YesStringThe path where the secret files will be mounted in the container.

Aerospike Configโ€‹

The YAML form of Aerospike server configuration. See Aerospike Configuration for details.

Rack Configโ€‹

Configures the operator to deploy a rack-aware Aerospike cluster. Pods are deployed in given racks based on the given configuration.

FieldRequiredTypeDefaultDescription
namespaces
Dynamic Rolling restart
NoList of StringsList of Aerospike namespaces for which rack feature will be enabled.
racks
Dynamic
YesList of structuresList of racks
rollingUpdateBatchSize
Dynamic
NoInteger or StringConfigures the number/percentage of rack pods that will be restarted simultaneously during rolling restart. It's value can be integer eg. 5 or string eg. "25%"

See Rack awareness for details.

Rackโ€‹

Specifies single rack config

FieldRequiredTypeDefaultDescription
idYesIntegerIdentifier for the rack.
zoneNoStringCloud provider specific Zone name for setting rack affinity. Rack pods will be deployed to the given cloud Zone.
regionNoStringCloud provider specific Region name for setting rack affinity. Rack pods will be deployed to the given cloud Region.
rackLabelNoStringRack label for setting rack affinity. Rack pods will be deployed in k8s nodes having rack label aerospike.com/rack-label: <rack-label>.
nodeNameNoStringK8s Node name for setting rack affinity. Rack pods will be deployed on the given k8s Node.
aerospikeConfig
Dynamic Rolling restart
NoStructureThis local AerospikeConfig is a patch, which will be merged recursively with common global AerospikeConfig and will be used for this Rack. See merge AerospikeConfig. If this AerospikeConfig is not given then global AerospikeConfig will be used.
storage
Dynamic Rolling restart
NoStructureThis local Storage specify persistent storage to use for the pods in this rack. If this Storage is not given then global Storage will be used.
podSpec
Dynamic Rolling restart
NoStructurePod overrides for this rack. See Rack Pod Overrides for details.

Rack Pod Overridesโ€‹

Provides rack-specific overrides to the Pod spec.

The following overrides are supported.

FieldRequiredTypeDefaultDescription
affinityNoKubernetes Pod AffinityKubernetes Affinity rules for pod placement. These rules will be merged with affinity rules generated by the operator.
tolerationsNoKubernetes Pod TolerationsKubernetes Toleration for Aerospike pod placement
nodeSelectorNoMap from string to stringNode selector constraints for the Aerospike pods.

Seeds Finder Servicesโ€‹

Creates additional Kubernetes service that allow clients to discover Aerospike cluster nodes.

FieldRequiredTypeDefaultDescription
loadBalancerNoStructureCreates a load balancer service that allows Aerospike clients to discover Aerospike cluster nodes. See #load balancer Service for details.

Load Balancer Serviceโ€‹

Creates a load balancer service which lets Aerospike clients discover Aerospike cluster nodes.

FieldRequiredTypeDefaultDescription
externalTrafficPolicyNoEnum - Local, ClusterExternal Traffic Policy Type string. See ServiceExternalTrafficPolicyType for details.
annotationsNoMap from annotation name to its valueKubernetes Annotations for the load balancer.
portNoIntegerExposed port on load balancer. If not specified targetPort is used.