# Configuring external targets

It is a best practice to configure at least one external target and perform backups regularly to minimize any downtime or data loss in failure scenarios.

{% hint style="info" %}
**Please Note:**

The external connections bypass proxy settings, even if a proxy is configured.
{% endhint %}

## SCP and SFTP External Targets

For either SCP or SFTP external targets, SSH authentication must be configured.

At the Brain's CLI as the `vectra` user run the following:

```language-markup
show backup public-key
```

Example output:

```language-markup
ssh-rsa AAAAB3NzaC1yc2EAAAADAobsfuscated8GPHFpZVYZTQYqcagJKJnnGDXAfbnXLTJ9ILM4JlR6ykAsRTZ8vobsfuscatedtA85I0kgyRtms4JhNVLyBS7cjeobsfuscated+hoIBBXNobsfuscatedUfbDjjlaSPEpzpustKjEIj3obsfuscatedxIEzKNbmrObpSt3gavFuSJslSbLM0+f9FZKobsfuscatedCkCa5OlDNe9GUmxe7JtzgnBAKrVmpt3uGrtsyxgr12eBYlthrpi4/bvlF9vs04KobsfuscatedN28WJyCaeygQleafvC/mobsfuscateda7ywZz
```

{% hint style="info" %}
**Please Note:**

This is an RSA key generated on-box for this use and is **NOT** the same as the SSL keys that can be provided by customers for use with the QUX UI served from the Brain appliance. This key cannot be changed by the customer.
{% endhint %}

On the receiving SCP or SFTP server:

* Paste that key in the `~/.ssh/authorized_keys` file.
* Ensure the permissions of the `~/.ssh` directory are not too open (e.g. sudo chmod 700 \~/.ssh).
* Ensure the permissions of the `~/.ssh/authorized_keys` file are not too open
  * For example you could: `sudo chmod 600 ~/.ssh/authorized_keys`

Then, follow the below instructions to set up the respective type of external target.

### SCP External Targets

Back on the brain, use the `backup external-targets configure scp` command to create a new external target for your SCP server. For example, for `ubuntu@192.168.12.34:/home/backups/`, the command would be:

```language-markup
backup external-targets configure scp --user ubuntu --server 192.168.12.34 --path /home/backups/
```

After setting the SCP target, the connection can be tested by:

```language-markup
backup external-targets test
```

Example output:

```language-markup
Success: SCP Target ubuntu@192.168.12.34:/home/backups
> Test copy to file server successful. Check for a file named "sched_backup_testo0an79lx" on the target.
```

{% hint style="info" %}
**Please Note:**

All configured external targets will be tested when using the above command.
{% endhint %}

By default, no limit will be set to the backups pushed to this target. To enable rotation after a certain number of backups, see the section on the `--max-backups` option below.

To enable weekly scheduled backs with external upload:

```language-markup
backup schedule enable --enable-external-upload
```

### SFTP External Targets

Back on the brain, use the `backup external-targets configure sftp` command to create a new external target for your SCP server. For example, for ubuntu\@192.168.12.34:/home/backups/, the command would be:

```language-markup
backup external-targets configure sftp --user ubuntu --server 192.168.12.34 --path /home/backups/
```

After setting the SFTP target, the connection can be tested by:

```language-markup
backup external-targets test
```

Example output:

```language-markup
Success: SFTP Target ubuntu@192.168.12.34:/home/backups
> Test copy to file server successful. Check for a file named "sched_backup_testmfb6u3kt" on the target.
```

{% hint style="info" %}
**Please Note:**

All configured external targets will be tested when using the above command.
{% endhint %}

By default, no limit will be set to the backups pushed to this target. To enable rotation after a certain number of backups, see the section on the `--max-backups` option below.

To enable weekly scheduled backs with external upload:

```language-markup
backup schedule enable --enable-external-upload
```

### Password-based SFTP Authentication

Alternatively, SFTP servers may use password-based SFTP authentication, instead of public-key.

In that case, pass the `--use-password` flag while configuring your SFTP target:

```language-markup
backup external-targets configure sftp --user ubuntu --server 192.168.12.34 --path /home/backups/ --use-password
```

You will be prompted to input the SFTP password. Note: this password will be encrypted via AES, then stored in the appliance database. The key used to encrypt is stored locally on-box with minimal access permissions.

## AWS S3 External Targets

Customers can configure an AWS S3 bucket to upload backups to.

### Requirements

* AWS IAM account with the following permissions:
  * `s3:PutObject`, `s3:DeleteObject`, `s3:GetObject`, `s3:ListBucket`, `s3:GetBucketPolicy`, `iam:SimulatePrincipalPolicy`
* Programmatic access to AWS S3 for this IAM user with:
  * AWS Access Key ID
  * AWS Secret Access Key
  * See [Managing access keys for IAM users](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) in AWS documentation for more details.
* AWS S3 bucket with policy permissions that allows the same S3 actions as described above for the AWS IAM account.

### **Configuring the S3 Bucket Policy Permissions**

Expand for details

* Navigate in your AWS account to S3.
* Choose the relevant bucket on which you want to apply the required permissions. Once inside, click on **Permissions**, scroll down to **Bucket policy** and click on **Edit**.

![](https://4227135129-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHJ1ltuWFvsArFWtevnRn%2Fuploads%2Fgit-blob-7df812af6f820e6b3f4f9dc6b35167ab00382388%2F5d6483ab570595eaeae9c6b8ec4d12a67720a121ef6b67b010d32bf1bdfd58be.jpg?alt=media)

* Paste the following code inside the policy. Replace the **XXX** with your bucket's name.
  * If you don't have an existing policy, paste this code inside the editor. Otherwise, add this code to the bottom of the page. This is only an example, the required permissions are: `s3:PutObject`, `s3:DeleteObject`, `s3:GetObject`, `s3:ListBucket`, `s3:GetBucketPolicy`.

```language-markup
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::XXX:user/search-archive-restore-user" # replace XXX with the ARN of your IAM user
            },
            "Action": [
                "s3:GetObject",
                "s3:ListBucket",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:GetBucketPolicy"

            ],
            "Resource": [
                "arn:aws:s3:::XXX",  # replace XXX with your bucket's name
                "arn:aws:s3:::XXX/*"  # replace XXX with your bucket's name
            ]
        }
    ]
}
```

{% hint style="info" %}
**Please Note:**

Please remove all comments `#...` from the lines in JSON above or it will fail to save it.
{% endhint %}

* Click on **Save changes** in the bottom right to apply the new policy. It might take a few minutes for the new policy/updates to take effect.

### Configuring the S3 target at the Brain CLI

```language-markup
backup external-targets configure s3 --region us-west-2 --bucket-name test-bucket-8dc3acae09c84502a482c4b784abecb8 --access-key-id KWE9LDKFTROULUJV1234
```

* You will be prompted to enter the AWS IAM account's secret access key.
  * Alternatively, provide it with the `--secret-access-key <ACCESS_KEY>` parameter.

{% hint style="info" %}
**Please Note:**

The secret access key will be encrypted via AES and then stored locally in the Brain's database. The key use to encrypt is stored locally with minimal access permissions.
{% endhint %}

* For a bucket that has a nested directory structure, pass a path to the backups folder with the `--path path/to/backups` parameter. See the [Backup Location in Bucket](#backup-location-in-bucket) section below for examples.

To configure an alternate authentication region for retrieving the STS token required to access the bucket:

* `--auth-region` is an optional parameter.
* Without it being specified, the specified `--region` for the bucket is still properly used to backup to S3, but the auth token is retrieved from us-east-1 because S3 handles global authentication through STS by default there.
* If you specify `--auth-region`, then STS token request goes to specified region.
  * For specific endpoints please see: <https://docs.aws.amazon.com/general/latest/gr/sts.html>
* In the example below, the auth region is set to eu-central-2:

```language-markup
backup external-targets configure s3 --bucket-name vectra-test-zurich --access-key-id [censored] --secret-access-key [censored] --region eu-central-2 --max-backups 2 --name "EUCentral2" --auth-region eu-central-2
```

After setting the S3 target, the connection can be tested by:

```language-markup
backup external-targets test
```

Example Output:

```language-markup
Success: S3 Target us-west-2 bucket test-bucket-8dc3acae09c84502a482c4b784abecb8, path .
> Test copy to file server successful. Check for a file named "sched_backup_test49cnbxz6" on the target.
```

{% hint style="info" %}
**Please Note:**

All configured external targets will be tested when using the above command.
{% endhint %}

To enable weekly scheduled backs with external upload:

```language-markup
backup schedule enable --enable-external-upload
```

### **Backup Location in Bucket**

By default, backups will be placed in the root directory of the bucket. For example:

```language-markup
# List of bucket objects
migration-8.4.0-0-10941-20240412-0022.tar.gz.gpg
migration-8.4.0-0-10941-20240419-0024.tar.gz.gpg
migration-8.4.0-0-10941-20240426-0021.tar.gz.gpg
```

For a nested bucket structure, use the `--path` parameter to specify a subdirectory. For this example structure:

```language-markup
/backups
    /brain-1
        migration-8.4.0-0-10941-20240412-0022.tar.gz.gpg
        migration-8.4.0-0-10941-20240419-0024.tar.gz.gpg
        migration-8.4.0-0-10941-20240426-0021.tar.gz.gpg
```

The `--paths` backups/brain-1 parameter should be used.

## Rotating Old Backups (--max-backups parameter)

{% hint style="info" %}
**Please Note:**

This section applies only to SCP, SFTP, and S3 targets.
{% endhint %}

For SCP, SFTP, and S3 targets, a `--max-backups` parameter may be passed during configuration. This will tell the brain to delete old backups on this target if the number of backups exceeds a certain count.

Example with an SCP target:

```language-markup
backup external-targets configure scp --user ubuntu --server 192.168.12.34 --path /home/backups --max-backups 5
```

Now, if the user runs a backup, the Brain will:

* Run the backup (locally).
* Check how many backup files are on this external target.
  * If more than 4 it will delete the oldest backup.
* Upload the backup.

When the backup complete there will be at most 5 backup files on this external target.

## Brain-to-Brain Backups

### Definitions, Facts, and Requirements

Setting another Brain as an external target can help you recover more quickly in the event that a failure is serious enough to cause the primary Brain to no longer function. Please keep in mind the following when using another Brain as an external target:

* The **source** Brain is the Brain where the backup is performed.
* The **target** Brain is the Brain the backup is copied to after it is completed on the source Brain.
* The target Brain stores the backup, but does not automatically restore the saved backup into its configuration.
  * In this manner the target brain can be considered as a cold spare rather than a hot spare and the target Brain could be used in its own right as a Brain if necessary.
  * If you plan to be able to restore the source backup to the target Brain, then the target Brain should only be used for backup storage because you would over write its configuration when restoring the backup to it.
* Source and target Brains can be of any type (physical, virtual, cloud).
* Source and target Brains can be in any mode that includes Brain functionality (Brain or Mixed, but not Sensor mode).
* Backup retention is managed automatically based on available disk space on the target Brain. Overall retention can therefore vary depending on individual backup sizes. Each time a backup is copied, previous backup files exceeding the maximum directory size (100GB on physical Brains, 20GB on virtual or cloud Brains) are deleted oldest-first until enough space is available for the new backup. For newly deployed brains you may see a larger amount of backups than a Brain which has been in production for a while. As Brains age, the backup file typically gets bigger and consequently the amount of backups will be reduce to avoid problems with disk space usage in the target Brain.
* Source and target Brains must be able to communicate to each other HTTPS and SFTP is used for the backup transfer. Please make sure firewall rules allow bidirectional communication over:
  * TCP/443
  * TCP/22
* The source Brain requires a token from the target Brain to be allowed to communicate with it for transfer of backups.
* The target Brain can further be protected by configuring an allow list that only allows communication from specific other hosts (Brains) in your environment.

### Configuring Brain-to-Brain Backup

A unique token is used to allow the source Brain to communicate with the target Brain. This token must be retrieved from the target Brain to complete the configuration on the source Brain.

On the target Brain:

```language-markup
show backup from-brain token
```

Example output:

```language-markup
This brain's token is: a2f7d42d0ece42ab6b57d2f44520d1e3d06e1234
This brain's IP is: 1.2.3.4
```

On the source Brain, use the obtained token and IP address to configure the backup target:

```language-markup
backup external-targets configure to-brain \
    --token a2f7d42d0ece42ab6b57d2f44520d1e3d06e1234 \
    --target-brain 1.2.3.4
```

Example output:

```language-markup
Created Brain-to-Brain Target 1.2.3.4 with name 'to-brain-1'
```

{% hint style="info" %}
**Please Note:**

The token will be encrypted via AES and then stored locally in the Brain's database. The key use to encrypt is stored locally with minimal access permissions.
{% endhint %}

To test your connection to the target Brain, from the source Brain execute the following:

```language-markup
backup external-targets test
```

{% hint style="info" %}
**Please Note:**

All configured external targets will be tested when using the above command.
{% endhint %}

Example output:

```language-markup
Success: Brain-to-Brain Target 1.2.3.4
> Brain connection check successful.
```

To enable weekly scheduled backs with external upload:

```language-markup
backup schedule enable --enable-external-upload
```

### Refreshing the Token

It is a good security practice to periodically refresh the token used for Brain-to-Brain backup. On the target Brain:

```language-markup
backup from-brain refresh-token
```

This will invalidate the old token and issue a new one. You will need to update the configuration on the source Brain with the new token. For example, if the target was named `to-brain-1`, run the following on the source Brain:

```language-markup
backup external-targets configure to-brain \
    --name to-brain-1 \
    --token replace_with_new_token \
    --target-brain 1.2.3.4
```

### Specifying an Allow List on the Target Brain

By default, any host on your network that presents the token may use Brain-to-Brain backups. You can restrict this by using the following command to set an allow list on the target brain:

```language-markup
backup from-brain allow-list 192.168.35.12 192.168.35.34 192.168.35.56
```

This will only allow 192.168.35.12, 192.168.35.34, and 192.168.35.56 to connect for Brain-to-Brain backups. To allow all hosts again, specify `--all-hosts`:

```language-markup
backup from-brain allow-list --all-hosts
```

To see the currently configured allow list:

```language-markup
show backup from-brain allow-list
```

## Testing, Renaming, and Removing External Targets

### Listing Targets

Unlimited external targets may be configured. Each target is identified by a human-readable name.

To see the external targets currently configured via the `show backup external-targets` command:

```language-markup
show backup external-targets
```

Example output:

```language-markup
scp-1: SCP Target vadmin@192.168.12.35:/home/backups
sftp-2: SFTP Target vadmin@192.168.12.36:/home/backups
to-brain-3: Brain-to-Brain Target 192.168.45.67
s3-4: S3 Target us-west-2 bucket test-bucket-8dc3aca82c4b784abecb8, path .
```

### Custom Target Names

When configuring any external target, the `--name` parameter can be used to set a custom name for the target. This parameter is not required, and will be set to the following by default:

* SCP targets: `scp-n`
* SFTP targets: `sftp-n`
* To-Brain targets: `to-brain-n`
* S3 targets: `s3-n`

`n` is the number of total targets configured at the time the target was named.

### Updating a Target

Use the name of the target to update its configuration. For example when using a custom name:

```language-markup
backup external-targets configure scp --user ubuntu --server 192.168.12.34 --path /home/backups --name my-scp-target
```

Example output:

```language-markup
Created SCP Target ubuntu@192.168.12.34:/home/backups with name 'my-scp-target'
...
```

Modifying to have `new_user` instead of `ubuntu`:

```language-markup
backup external-targets configure scp --user new_user --server 192.168.12.34 --path /home/backups --name my-scp-target
```

Example output:

```language-markup
Updated 'my-scp-target' to SCP Target new_user@192.168.12.34:/home/backups
...
```

### Testing Connection to a Target

Test connection to external targets with the backup external-targets test command:

```language-markup
backup external-targets test
```

Example output:

```language-markup
Success: SCP Target ubuntu@192.168.12.35:/home/backups
> Test copy to file server successful. Check for a file named "sched_backup_testo0an79lx" on the target.
Success: SFTP Target ubuntu@192.168.12.36:/home/backups
> Test copy to file server successful. Check for a file named "sched_backup_testmfb6u3kt" on the target.
Success: S3 Target us-west-2 bucket test-bucket-8dc3aca82c4b784abecb8, path .
> Test copy to file server successful. Check for a file named "sched_backup_test49cnbxz6" on the target.
Success: Brain-to-Brain Target 192.168.35.116
> Brain connection check successful.
```

Rename a target as follows:

```language-markup
backup external-targets rename --from scp-1 --to SCP-ubuntu-my-server
```

Example output:

```language-markup
External targets list is now:
1. SCP-ubuntu-my-server: SCP Target vadmin@192.168.12.35:/home/backups
2. sftp-2: SFTP Target vadmin@192.168.12.36:/home/backups
3. to-brain-3: Brain-to-Brain Target 192.168.45.67
4. s3-4: S3 Target us-west-2 bucket test-bucket-8dc3aca82c4b784abecb8, path .
```

### Removing a Target

To remove a target, use the following command (and use the name of the target you wish to remove, included below is a sample name):

```language-markup
backup external-targets remove --name SCP-ubuntu-my-server
```

Example output:

```language-markup
Removed.
External targets list is now:
1. sftp-2: SFTP Target vadmin@192.168.12.36:/home/backups
2. to-brain-3: Brain-to-Brain Target 192.168.45.67
3. s3-4: S3 Target us-west-2 bucket test-bucket-8dc3aca82c4b784abecb8, path .
```