rclone with WebShield S3

rclone is a versatile command-line tool for syncing files to and from object storage. It works with WebShield S3 through the generic S3 backend. See Object storage for how to obtain your keys and create buckets.

Prerequisites

• Endpoint: https://s3.webshield.pro
• Access Key ID and Secret Access Key from the S3 storage page in the control panel.
• The full bucket name including the account prefix (for example, u42-backups).

Install

# Debian/Ubuntu
sudo apt install rclone
# macOS
brew install rclone
# or the official installer: https://rclone.org/install/

Configure a remote

Add a remote to ~/.config/rclone/rclone.conf (create the file if it does not exist):

[webshield]
type = s3
provider = Other
access_key_id = WSS...
secret_access_key = <secret>
endpoint = https://s3.webshield.pro
force_path_style = true
no_check_bucket = true

provider = Other and force_path_style = true are what make rclone talk to a generic, path-style S3 endpoint. no_check_bucket = true suppresses the bucket creation attempt rclone makes before an upload; without it some commands fail with AccessDenied, see CreateBucket: Access Denied. Alternatively, run rclone config and pick Amazon S3 Compliant → Other, then enter the same values interactively.

Everyday commands

The remote name (webshield:) is followed by bucket/path:

rclone ls webshield:u42-backups                 # list objects
rclone copy ./dump.sql.gz webshield:u42-backups/db/   # upload one file
rclone sync ./media webshield:u42-backups/media       # mirror a directory (deletes extras)
rclone copy webshield:u42-backups/db ./restore        # download
rclone delete webshield:u42-backups/old               # delete a path

rclone sync makes the destination match the source, including deletions — use rclone copy if you only want to add or update files.

Useful flags:

rclone sync ./media webshield:u42-backups/media \
  --progress \            # live transfer stats
  --transfers 16 \        # parallel transfers
  --checksum              # compare by checksum, not modification time

CreateBucket: Access Denied

Uploading a single file may fail with this error even though the bucket exists and is writable:

ERROR : cli.rs: Failed to copy: failed to prepare upload: operation error S3:
CreateBucket, https response error StatusCode: 403, api error AccessDenied: Access Denied.

The fix is no_check_bucket = true in the remote section (see above), or the equivalent one-off flag:

rclone copyto --s3-no-check-bucket ./file.txt webshield:u42-backups/file.txt

The cause: before an upload rclone issues a defensive CreateBucket and expects BucketAlreadyOwnedByYou in return. WebShield keys are not allowed to create buckets — that is done only in the control panel — so the gateway answers AccessDenied and rclone treats the upload as impossible.

This also explains the asymmetry: copying a directory succeeds while copying a single file does not. For a directory rclone first lists the destination, which marks the bucket as known to exist, so CreateBucket is never sent. A single-file copy performs no listing, so the check fires.

Notes

• WebShield uses path-style addressing; force_path_style = true selects it.
• Buckets are created and deleted only in the control panel; rclone mkdir webshield:name cannot create a bucket.
• If the account balance goes negative, storage becomes read-only: copy/sync uploads fail while downloads keep working. Top up the balance to restore writes.