Estimate resources for Aerospike restore

This page describes how to estimate memory, disk space, and file descriptor usage when running asrestore.

Overview

asrestore can use significant resources, depending on the options that you set. We recommend that you run asrestore on a separate host or container from the Aerospike Database.

The following explanations and formulas describe how the tool uses memory, storage, and file descriptors.

The command line flags used in the following formulas are defined in Run Aerospike restore. These formulas describe how resource usage scales and they do not give exact answers.

Allocate slightly more resources to these tools than the formulas suggest.

Estimate memory usage for a restore

The following have the greatest impact on memory used by asrestore:

• The system where you are running asrestore
• The asrestore version
• The --parallel, --batch-size, and --max-async-batches flags

Formula to approximate memory usage for restore

To calculate the approximate amount of memory usage that will be used for a restore:

1. Multiply the values that you set for the --parallel and --batch-size flags.
2. Mutiply the result from the previous step by the value returned by avg_record_size.
3. Multiply the value from --batch-size by the value returned from --max-async-batches.
4. Add the result from the previous step to the result from Step 2.
5. Add the result from the previous step to the value from --max-async-batches.

The final result is the approximate amount of memory usage for a restore of your data.

This formula still holds even if --disable-batch-writes is set. In this case, all that changes with respect to the formula is the default --batch-size value.

Calculate number of file descriptors

asrestore may need to open many backup files and network sockets. It is important to allow the asrestore processes to open the proper number of file descriptors to prevent too many open file errors. Here are some ways to estimate the required file descriptors.

By default, asrestore opens a backup file for each of its --parallel threads. If asrestore is running against Database 6.0 or later, it uploads records in batches. In this case, asrestore opens up to --max-async-batches sockets to the server for record upload.

If the server version is older than 6.0 or the --disable-batch-writes flag is used, each record is uploaded individually, so the sum of the value from --batch-size multiplied by the value from --max-async-batches is the number of sockets that may be opened.

Formula to calculate approximate file descriptors for restore

To calculate the approximate number of file descriptors for a data restore with batch writes:

• Sum the values that you set for the --parallel flag and returned by the --max-async-batches flag.

To calculate the approximate file descriptors for a data restore without batch writes, perform the following steps:

1. Multiply the value from --batch-size by the value returned from --max-async-batches.

2. Add the result from the previous step to the value returned by the --parallel flag.

The final sum is the approximate number of file descriptors needed for a data restore.