Skip to main content
We are excited to be a part of AWS re:Invent 2024. Visit us at booth #1844 in Las Vegas.More info
Loading

Configure rack awareness

Overview​

This page describes how to implement rack awareness in namespaces configured for Available and Partition-tolerant (AP) mode.

Rack awareness stores replicas of records in separate hardware failure groups which are defined by their rack-id.

note

For instructions on implementing rack awareness in a namespace configured for strong consistency (SC), see Configure rack awareness .

How it works​

The following examples illustrate how rack awareness operates.

  • When configuring three racks with replication factor of 3 (RF3), each rack receives one replica for a given partition.
    • The three replicas are on separate racks. The distribution of specific nodes is specified in the succession list.
  • If you lose a rack, the number of replicas is eventually restored to match the value of your replication-factor. For instance:
    • your server is configured for RF3
    • you reduce the number of racks from 3 racks to 2
    • one rack hosts the master
    • the other rack hosts one replica
    • and the third replica moves to one of the two racks.
  • To avoid having data missing from the cluster, configure rack awareness to use multiple racks defined by rack-id.
  • Starting with Database 7.2, the active rack feature dynamically designates a particular rack-idto hold all master partition copies.

Partition distribution​

Masters are always evenly distributed across each node in the cluster, regardless of the configuration in prefer-uniform-balance, even when the number of racks is greater than the replication-factor (RF).
In these cases, racks do not have a copy of each partition.

Imbalanced racks​

Imbalanced racks are racks with different numbers of nodes. The master partition and replicas are distributed to specific racks, however if the RF is configured higher than the current number of racks, the extra replicas are distributed randomly.

Imbalanced clusters​

If a single node goes down, the cluster is temporarily imbalanced until the node is restored. This imbalance does not cause service interruptions. The cluster continues uninterrupted. Once the node restarts, the cluster automatically rebalances. The imbalance in the general load on the nodes across racks depends on the nature of the workload.

Implement rack awareness on an AP namespace​

You can configure rack awareness at the namespace level. To assign nodes to the same rack, specify the same rack-id for these nodes.

namespace {
    ...
    rack-id 1
    ...
}

Implement rack awareness dynamically on a cluster​

You can implement rack awareness for an existing cluster dynamically.

note

Tools package 6.0.x or later is required to use asadm's manage config commands. manage config requires asadm to be in enable mode by typing enable. Otherwise, use the equivalent asinfo - set-config command.

  1. On each node use asadm's manage config command to change the rack-id to the desired value: 
asadm -e "enable; manage config namespace <namespaceName> param rack-id to 1 with <host>"

or use asinfo's set-config command:

asinfo -v "set-config:context=namespace;id=namespaceName;rack-id=1"
  1. Add the rack-id to the namespace stanza in the configuration file to ensure that the configuration persists following any restarts.
  2. Trigger a rebalance of the cluster to engage migrations with the new rack-id configurations.
asadm --enable -e "manage recluster"
note

Make sure to persist your changes in the configuration file to protect against a rollback due to restart. Also, verify that there are no typos in the configuration file. This is the best practice for updating the configuration file for any dynamic change.

Display rack information​

Use the following command to display the rack information.

note

Tools package 6.2.x or later is required to use asadm's show racks command. Otherwise, use the equivalent asinfo - racks command.

asadm -e "show racks"
~~~~~~~~~~~~~~~~Racks (2021-10-21 20:33:28 UTC)~~~~~~~~~~~~~~~~~
Namespace|Rack|                                            Nodes
         |  ID|                                                 
bar      |2   |BB9040016AE4202, BB9020016AE4202, BB9010016AE4202
test     |1   |BB9040016AE4202, BB9010016AE4202                 
Number of rows: 2

For the test namespace, rack-id 1 includes nodes BB9040016AE4202, BB9020016AE4202, BB9010016AE4202. rack-id 2 includes nodes BB9040016AE4202, BB9010016AE4202.

Configure the node-id​

Partition distribution is based on the cluster's node-id. Node IDs can be changed one node at a time in a rolling fashion across a cluster.

Specify the node ID inside the server stanza of the configuration file, as shown in the following example.

```
service {
    <...>
    node-id a1
    <...>
}
```

Rack awareness reads​

Database clients can read from the servers in the closest rack or zone on a preferential basis. This can lower latency, increase stability, and significantly reduce traffic charges by limiting cross-availability-zone traffic.

This feature is available in Java, C, C#, Go and Python clients.

Set up rack aware reads​

  1. Set up clusters in logical racks. (See Implement rack awareness on an AP namespace.)

  2. Set the rackId and rackAware flags in the ClientPolicy object. Use the rack ID specified in the nodes for the associated-AZ where that application is running. The following example uses Java to demonstrate how to enable rack awareness. Operations are similar in other clients.


    ClientPolicy clientPolicy = new ClientPolicy();
    clientPolicy.rackId = <<rack id>>;
    clientPolicy.rackAware = true;

note

To avoid hard-coding, the rack ID can be obtained dynamically using the cloud provider's API, or set in a local property file.

  1. Once the application has connected, set 2 additional parameters in the Policy associated with the reads to be rack aware.

    AP Mode

    Policy policy = new Policy();
    policy.readModeAP = ReadModeAP.ALL;
    policy.replica = Replica.PREFER_RACK;
    readModeAP.ALL indicates that all replicas can be consulted. 
    policy.replica = Replica.PREFER_RACK indicates that the record in the same rack should be accessed if possible.

Designate an active rack​

When an active rack is designated, statically or dynamically, a particular rack ID will hold all master partition copies. For active rack to take effect, all nodes must agree on the same active rack, and the number of racks must be at most equal to the configured replication-factor.

active-rack 0 disables the feature. This means that you can’t designate rack_id 0 as the active rack.

Changing the rack ID on all nodes with rack_id 0 to a new value that is distinct from any other racks does not cause any migrations.

Enable active rack in aerospike.conf​

namespace ns-name {
    ...
    rack-id X
    active-rack Y  # Y may be the same as X
    ...
}

Enable active rack using asadm​

Admin> enable
Admin+> manage config namespace ns-name param active-rack to 1
~Set Namespace Param active-rack to 1~
            Node|Response
172.22.22.1:3000|ok      
172.22.22.2:3000|ok      
172.22.22.3:3000|ok      
172.22.22.4:3000|ok      
172.22.22.5:3000|ok      
172.22.22.6:3000|ok      
Number of rows: 6

Where to next?​