Use All-Flash for Aerospike on Kubernetes

danger

For Aerospike server version < 6.3, all flash requires the Aerospike pod to be run in elevated privileged mode. In privileged mode when using index on flash, Aerospike Server tweaks the Kubernetes worker node's sysctls for Virtual Memory.

See privileged: true in the podSpec below.

For Aerospike server version >= 6.3, all flash can run in unprivileged mode if all the required flash kernel parameters are set on all the Kubernetes nodes. Refer Linux best practices for All-Flash deployment

This example creates a cluster where Aerospike data and index both use flash storage. For more information on all-flash and other storage configurations, see the Aerospike documentation for namespace storage configuration.

caution

There is an issue with Kubernetes versions 1.21 and 1.22 where volumeDevices mapping is ignored when containers run in privileged mode.

This bug does not affect Kubernetes v1.23+ and containerd v1.15.10+ in our testing. We recommend the same or higher versions of Kubernetes and containerd for running All-Flash Aerospike clusters.

To set this up, see the example Aerospike CR below:

apiVersion: asdb.aerospike.com/v1
kind: AerospikeCluster
metadata:
  name: aerocluster
  namespace: aerospike

spec:
  size: 2
  image: aerospike/aerospike-server-enterprise:7.1.0.0

  storage:
    filesystemVolumePolicy:
      initMethod: deleteFiles
      cascadeDelete: true
    blockVolumePolicy:
      cascadeDelete: true
    volumes:
      - name: workdir
        aerospike:
          path: /opt/aerospike
        source:
          persistentVolume:
            storageClass: ssd
            volumeMode: Filesystem
            size: 1Gi
      - name: ns
        aerospike:
          path: /test/dev/xvdf
        source:
          persistentVolume:
            storageClass: ssd
            volumeMode: Block
            size: 5Gi
      - name: ns-index
        aerospike:
          path: /test/dev/xvdf-index
        source:
          persistentVolume:
            storageClass: ssd
            volumeMode: Filesystem
            size: 5Gi
      - name: aerospike-config-secret
        source:
          secret:
            secretName: aerospike-secret
        aerospike:
          path: /etc/aerospike/secret

  podSpec:
    multiPodPerHost: true
    aerospikeContainer:
      securityContext:
        privileged: true

  aerospikeAccessControl:
    users:
      - name: admin
        secretName: auth-secret
        roles:
          - sys-admin
          - user-admin
          - data-admin
          - read
          - write

  aerospikeConfig:
    service:
      feature-key-file: /etc/aerospike/secret/features.conf
    security: {}
    network:
      service:
        port: 3000
      fabric:
        port: 3001
      heartbeat:
        port: 3002
    namespaces:
      - name: test
        replication-factor: 2
        index-type:
          type: flash
          mounts:
            - /test/dev/xvdf-index
          mounts-budget: 4294967296
        storage-engine:
          type: device
          devices:
            - /test/dev/xvdf

For the full CR file, see the example all-flash cluster CR.

This and other example CRs are stored in the main Aerospike Kubernetes Operator repository.

Save and exit the CR file, then use kubectl to apply the change.

kubectl apply -f aerospike-cluster.yaml