Skip to main content
Aerospike 8 introduces a new era of real-time transaction processingLearn more
Loading

Estimate resources for Aerospike backup

This page describes how to estimate memory, disk space, and file descriptor usage when running asbackup.

Overview​

asbackup can use significant resources. We recommend that you run the tool on a separate host or container from Aerospike database.

asbackup resource consumption depends on the options that you set when running the tool.

See the explanations and formulas below for how the tool uses memory, storage, and file descriptors. The command line flags used in the following formulas are defined in Run Aerospike backup.

note

The information outlined here applies toasbackup 3.11.0 and later.

These formulas describe how resource usage scales. They do not provide exact numbers for resource usage. You should allocate slightly more resources to asbackup than the formulas suggest.

Estimate memory usage for a backup​

asbackup's memory usage is largely affected by the --parallel flag. By default, asbackup distributes work across threads by assigning each thread a unique file to write to. These files each have a constant size buffer associated with them. The buffer in asbackup is 4KiB.

Formula to approximate memory usage for backup​

To calculate the approximate amount of memory usage that will be used for a backup:

  • Multiply the value that you set for the --parallel flag by 4KiB.

The result is the approximate amount of memory needed to back up of your data.

If you are backing up to Amazon S3, more memory will be used to maintain the internal S3 client and buffers.

Estimate disk space for a backup​

To estimate how much disk space is needed to back up your data, use the --estimate flag when running asbackup. Using the --directory and --output-file flags is not necessary.

When you use the --estimate flag, you may not use --parallel. The two flags are mutually exclusive.

asbackup --namespace NAME --estimate

The --estimate flag reads 10,000 records from the specified namespace and prints the average size of the sampled records.

Formula to estimate disk space for a backup​

To estimate the amount of disk space needed for a backup, perform the following steps:

  1. Multiply the estimated record size, returned by the --estimate flag, by the number of records in namespace.

  2. Add 10% of estimated record size.

The result is the approximate disk space needed for a backup.

note

Per-record filters (filter-exp, modified-after, modified-before, no-ttl-only, after-digest, and partition-list) and node-list are not accounted for in the estimate, and using these options will have no effect on the estimate.

Calculate number of file descriptors​

asbackup may need to open many backup files and network sockets. If asbackup is unable to open the required number of file descriptors to prevent too many open file errors. Here are some ways to estimate the required file descriptors.

By default, asbackup opens a new backup file for each of its --parallel threads. Each thread may have to open a network socket to each node in the cluster.

Formula to calculate approximate file descriptors for backup​

To calculate the approximate number of file descriptors needed for a backup:

  1. Multiply the value that you set for the --parallel flag by the value returned from nodes_in_cluster.

  2. Add the result of the previous step to the value returned from the backup_file_count flag.

In this calculation, backup_file_count is equal to --parallel when using --directory.

The final sum is the approximate number of file descriptors needed for a backup.