S3 buckets

With s3.enabled, the cluster backup mounts the buckets listed in cluster.s3Buckets with s3fs and archives them with its other sources. The s3 values are endpoint, region (SigV4 region; empty = s3fs's us-east-1, which servers that check the region, such as Garage, refuse), mountPath, credentials, verifyListing and onMountFailure; the variables they set are listed under Environment variables.

A bucket that cannot be mounted

A bucket that cannot be listed within 30 seconds after mounting is reported with the bucket, endpoint and region. With onMountFailure: skip (default) the start goes on without it and stops only when no bucket can be mounted (a wrong endpoint, region or credentials); each cluster backup then ends with a warning that names the bucket (in plan mode through the pre-backup hook s3-check-mounts, as a warning in Borg UI; the hook continues on error, so the other sources are backed up even when no bucket is mounted). With onMountFailure: fail one such bucket stops the start.

The listing check

Before each cluster backup, verifyListing (default on) compares each mount with the bucket's listing through the S3 API; a mount that misses objects, or that is not mounted, is mounted again, and one that still misses them fails the run.

Plan mode

In plan mode the console pod's mounts live as long as the pod, so each plan run first mounts every bucket fresh (pre-backup hook s3-remount-buckets, the same mount and policy as at the start), then runs the check as the hook s3-verify-listing, ahead of s3-check-mounts, and a bucket that still misses objects ends the run with a warning in Borg UI (Borg UI has no way to fail a plan run after its backup).