Managing sets in a namespace
Overview
This page describes how to manage sets in a namespace. An Aerospike set is similar to a table in a relational database, except you do not need to define a schema for a set. Sets are created dynamically and implicitly on the first record insertion in set. For more see Data Model.
Assigning a default TTL to a set
As of Database 7.0, you can specify a default time-to-live (default-ttl) configuration option
at the set level, which overrides any default-ttl option specified at the namespace level.
The asadm command to dynamically
specify default-ttl is:
asadm -e "enable; manage config namespace test set demo param default-ttl 60D"
See Data Retention for more details about record TTL.
Protecting a set from evictions
A set can be protected from evictions with a dynamic configuration. If evictions are in effect for a namespace, sets with evictions disabled will be skipped, regardless of the void-time of the records.
- Starting with Database 5.6.0, use the configuration parameter
disable-eviction - In Database 3.6.1 to 5.5.0, use the configuration parameter
set-disable-eviction
The asadm command to dynamically disable set evictions is:
asadm -e "enable; manage config namespace test set demo param disable-eviction to true"
Or using the older info command:
asadm -e "enable; asinfo -v 'set-config:context=namespace;id=test;set=demo;disable-eviction=true' "
Configuration parameters are applied per-node. asadm sends the same info command disabling evictions to all the nodes in the cluster.
Capping the size of a set
Starting with Aerospike 6.3, you can use the stop-writes-size configuration parameter to specify a size limit for a set.
If this threshold is breached, client writes to the set are refused. Deletions, replica writes, and migration writes are still allowed when the set is in stop-writes mode.
Once the limit is reached, the server does not allow any additional writes, even those that would decrease the size of a record. There are two ways to get under the stop-writes-size limit: increase or remove the limit, or delete records. However, record deletions using UDF, delete-all-bins ops, or background ops queries will also be rejected – only regular record deletes (including batch deletes) and nsup deletes will be allowed.
The asadm command to dynamically cap the size of a set is:
asadm -e "enable; manage config namespace test set demo param stop-writes-size to 250M"
Or using the older info command:
asadm -e "enable; asinfo -v 'set-config:context=namespace;id=test;set=demo;stop-writes-size=250M' "
Configuration parameters are applied per-node. asadm sends the same info command limiting the set size to all the nodes in the cluster.
The cluster-wide size quota for the set is number-of-cluster-nodes x stop-writes-size.
The threshold is checked and stop-writes triggered independently on each node.
Use show stop-writes to view the configured cap and to see how close you are to crossing the limit.
Tools package 8.4.x or later is required to use `show stop-writes.
Capping the number of records in a set
A set can be capped to a maximum number of records. If this threshold is breached, client writes to the set are refused. Deletions, replica writes, and migration writes are still allowed when the set is in stop-writes mode.
- Starting with Aerospike 5.6.0, use the configuration parameter
stop-writes-count - In Aerospike 3.6.1 to Aerospike 5.5.0, use the configuration parameter
set-stop-writes-count
The asadm command to dynamically cap the number of records in a set is:
asadm -e "enable; manage config namespace test set demo param stop-writes-count to 1000000"
Or using the older info command:
asadm -e "enable; asinfo -v 'set-config:context=namespace;id=test;set=demo;stop-writes-count=1000000' "
Configuration parameters are applied per-node. asadm sends the same info command limiting the number of records in the set to all the nodes in the cluster.
The cluster-wide record-count quota for the set is number-of-cluster-nodes x stop-writes-count.
The threshold is checked and stop-writes triggered independently on each node.
Use show stop-writes to view the configured cap and to see how close you are to crossing the limit.
Tools package 8.4.x or later is required to use `show stop-writes.
Adding and Removing a Set Index
Introduced in Aerospike Database 5.6, a set index
can be dynamically configured in the set context through declaring
enable-index to
be true. This may significantly improve performance when scanning the records in
the set.
asadm -e "enable; manage config namespace test set demo param enable-index to true"
Or using the older info command:
asadm -e "enable; asinfo -v 'set-config:context=namespace;id=test;set=demo;enable-index=true' "
Removing the set index is simply a matter of changing the value to
false, which is also the default for each set.
asadm -e "enable; manage config namespace test set demo param enable-index to false"
Or using the older info command:
asadm -e "enable; asinfo -v 'set-config:context=namespace;id=test;set=demo;enable-index=false' "
Monitoring Set Indexes
The memory used for all set indexes in a namespace is available using the
memory_used_set_index_bytes
namespace statistic and also using the
"memory-usage"
server log ticker line.
Clearing a Set
The total number of sets in a namespace is limited to 1023 in Database 6.0 and earlier, and 4095 in Database 7. Deleting a set metadata to reclaim space from this limit requires durably deleting all the data in the set and cold restarting the node. See How to clear up set names when they exceed the limit for details.
Truncating a Set in a Namespace
To initiate a set truncation, the asadm, asinfo, or aql
tools can be used to issue a truncate command to one Aerospike cluster node at a time.
To truncate a set, use either of the following commands.
Using the asadm command-line interface:
asadm
Seed: [('127.0.0.1', 3000, None)]
Aerospike Interactive Shell, version 2.11.0
Found 1 nodes
Online: 127.0.0.1:3000
Admin> info set
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Set Information (2023-03-26 22:59:40 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~
Namespace| Set| Node| Memory| Disk|~~~~~Quota~~~~~~| Objects| Stop| Disable| Set
| | | Used| Used| Total|Used%| | Writes|Eviction|Index
| | | | | | | | Count| |
test |demo|127.0.0.1:3000 |1.236 MB|0.000 B |250.000 MB|0.0 %|99.711 K|1000000|True |No
test |demo| |1.236 MB|0.000 B |250.000 MB|0.0 %|99.711 K| | |
Number of rows: 1
Admin> enable
Admin+> manage truncate ns test set demo
You're about to truncate up to 0 records from set demo for namespace test
Confirm that you want to proceed by typing 6cdc8c, or cancel by typing anything else.
6cdc8c
Successfully started truncation for set demo of namespace test
Admin+> info set
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Set Information (2023-03-26 23:04:08 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~
Namespace| Set| Node| Memory| Disk|~~~~~Quota~~~~~~|Objects| Stop| Disable| Set
| | | Used| Used| Total|Used%| | Writes|Eviction|Index
| | | | | | | | Count| |
test |demo|127.0.0.1:3000 |0.000 B |0.000 B |250.000 MB|0.0 %|0.000 |1000000|True |No
test |demo| |0.000 B |0.000 B |250.000 MB|0.0 %|0.000 | | |
Number of rows: 1
Or in non-interactive mode:
asadm --enable -e "manage truncate ns test set demo --no-warn"
Or with asinfo send truncate a single node. The truncate operation is then
communicated to the other nodes as shared metadata (SMD):
asinfo -h 127.0.0.1 -v "truncate:namespace=test;set=demo"
Mechanism
The mechanism involved in truncating a namespace or set within a namespace is distinct from NSUP.
Before Database 6.3, the truncate info command started a single background thread on each of the nodes in the cluster.
From Database 6.3, the truncate info command starts a number of threads for each truncation job equal to the value of the truncate-threads configuration parameter. The truncate threads then traverse the primary index of the namespace and remove the necessary records.
From Database 6.3, set truncation is optimized to use set indexes to rapidly find all the records belonging to the set being truncated. This optimization significantly increases the speed of set truncation, with the caveat that it is skipped if the set has tombstones.
The truncate command takes an optional last update time (LUT) threshold parameter, lut which limits the truncation to records with an LUT earlier than the specified timestamp . If no timestamp is specified, the "now" timestamp of the receiving node is used.
Set truncation can be issued repeatedly. It updates the effective last update time of an affected set, and is reflected in the lut field of the set information.
For truncation on sets with secondary indexes, each record data is actively read, and any secondary index entries associated with the record are also removed. See Manage sindexes.
In the Enterprise Edition, truncation is durable and preserves record deletions through a cold-restart. In the Community Edition, similar to record deletes, records in previously truncated sets are not durable and deletes can return through a cold start.
The truncate-undo and
truncate-namespace-undo commands can be used to remove the truncate related entry(ies) in SMD to allow
for potential recovery of inadvertently truncated records upon a subsequent cold restart. Truncated records that would have been already overwritten on persistent storage would not be recoverable.
During cluster change
If nodes return to the cluster during or after a truncate or truncate-namespace command has run, the SMD subsystem provides information on what has been truncated. The nodes will match the truncation level of the rest of the cluster by truncating data based on the LUT received through the SMD subsystem.
During migrations, records are checked against the truncation criteria based on the timestamp saved in the SMD truncate file for the specific set and/or namespace as partitions migrate across nodes. If they meet the truncation criteria, the records are discarded. In this way, truncate and truncate-namespace are completely robust during cluster memberships changes.
When nodes are executing truncate or truncate-namespace, incoming transactions are only executed if the LUT of the accessed record is older than the execution time of any truncation command (pending or completed). From an application's perspective, as soon as a truncation command starts, whether directly, through an SMD update, or when a node joins a cluster, the would-be truncated record cannot be differentiated from an already truncated record, and the command works as if it were instantaneous. This is possible as records accessed through any transactions are checked against the LUTs stored in the SMD subsystem.
Manage sindexes
The presence of sindexes can slow down truncation. Delete any sindexes before running truncate or truncate-namespace, as described in the following table.
| Storage Engine | Versions | Description |
|---|---|---|
| All Flash | Prior to Database 5.7.0.20 | Before you truncate, delete the secondary index (sindex) associated with the set to avoid performance impact. |
| Database 6.0 prior to 6.0.0.4 | Before you truncate, delete the sindex associated with the set to avoid performance impact. | |
| Database 5.7.0.20 and later and Database 6.0.0.4 and later | Do not delete sindex before truncating. Truncation updates the secondary index on the fly. | |
| Other storage engines (including persistent memory, data-in-memory, data-in-index, memory-only, and shadow device) | Database prior to 5.7 | Before you truncate, delete the sindex associated with the set to avoid performance impact. |
| Database 5.7 and later | Do not delete any sindex prior to truncating a set. Deleting a secondary index while a truncation is in progress could impact performance. |
If using the client APIs to perform the truncation, the LUT (last update time) is accurate to 1 millisecond (ms) resolution.