CyberPanel Backups v2 Proposal

Yes rlcone config file, while doing backup of emails, the user does not have access to rclone backend file.

When the command runs as root it obviously can’t find the rclone backend as rlcone looks into the user directory, thus if we can pass the path of config file manually, it will solve the issue.

Just saw that rclone seems to support the XDG_CONFIG_HOME and RCLONE_CONFIG env variables.

So, you can either set XDG_CONFIG_HOME which would use $XDG_CONFIG_HOME/rclone/rclone.conf or directly specify the config file in the env variable RCLONE_CONFIG.

Still, I’ll open an issue to let rustic set rclone options.

Did you manage to add it by any chance?

@usmannasir You just have to set the RCLONE_CONFIG environment variable. I think this already works with all rustic versions.

If multiple backups run at same time, this can cause conflict.

Can you describe what kind of conflicts you mean?

Sorry got caught up in something, I manage to set the env variable, thanks alot.

Is it possible to use hetzner storage boxes for backups?

1 Like

We had voting in our Facebook groups, hetzner storage box didnt got much votes, but in later releases we can include it.

We can use hetzner storage boxes as a SFTP destination, but it gives us errors.

What will work now? s3? what if I mount Storage Box via Samba/CIFS?

I will create a video tutorial around it, basically as of this current release you can configure two sort of rclone backends from gui. However virtually you can configure any rclone backend manually from cli and CyberPanel will detect it and it will work just like a backup configured from UI.

1 Like

I was doing my final tests and ran into this

root@v2tests:~# sudo -u vtest8202 rustic -r rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website snapshots
using no config file (./rustic.toml doesn't exist)
enter repository password: 
[INFO] repository rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website: password is correct.
[INFO] using cache at /home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba

snapshots for (host [v2tests], label [], paths [/home/backup/v2testbackup.oso5ecswoq.cyberpanel.website/config.json,/home/v2testbackup.oso5ecswoq.cyberpanel.website,vB8ajaWbNO67rx.sql])
| ID       | Time                | Host    | Label | Tags | Paths                         | Files | Dirs |     Size |
|----------|---------------------|---------|-------|------|-------------------------------|-------|------|----------|
| 4054489e | 2023-04-11 21:48:15 | v2tests |       |      | /home/backup/v2testbackup.oso |  3275 |  400 | 88.7 MiB |
|          |                     |         |       |      | 5ecswoq.cyberpanel.website/co |       |      |          |
|          |                     |         |       |      | nfig.json                     |       |      |          |
|          |                     |         |       |      | /home/v2testbackup.oso5ecswoq |       |      |          |
|          |                     |         |       |      | .cyberpanel.website           |       |      |          |
|          |                     |         |       |      | vB8ajaWbNO67rx.sql            |       |      |          |
| 25d98af0 | 2023-04-11 22:01:42 | v2tests |       |      | /home/backup/v2testbackup.oso |  3285 |  412 | 89.1 MiB |
|          |                     |         |       |      | 5ecswoq.cyberpanel.website/co |       |      |          |
|          |                     |         |       |      | nfig.json                     |       |      |          |
|          |                     |         |       |      | /home/v2testbackup.oso5ecswoq |       |      |          |
|          |                     |         |       |      | .cyberpanel.website           |       |      |          |
|          |                     |         |       |      | vB8ajaWbNO67rx.sql            |       |      |          |
2 snapshot(s)

Then while i try to dump the database

root@v2tests:~# sudo -u vtest8202 rustic -r rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website dump 4054489e:vB8ajaWbNO67rx.sql --password ""
using no config file (./rustic.toml doesn't exist)
[INFO] repository rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website: password is correct.
[INFO] using cache at /home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba
[00:00:00]                                β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ          0/0                      [INFO] getting snapshot...
[00:00:00] reading index...               β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ          8/8                      Error: 

"vB8ajaWbNO67rx.sql" not found

it says vB8ajaWbNO67rx.sql not found, although its present in the snapshot.

Can you run rustic ls 4054489 to check the contents of this snapshot?
If in the output vB8ajaWbNO67rx.sql is missing: Can you tell me which command(s) you used to create that snapshot?

@aawsome

command used for backup

mysqldump --defaults-extra-file=/home/cyberpanel/.my.cnf -u root --host=localhost --port 3306 --add-drop-table --allow-keywords --complete-insert --quote-names --skip-comments vB8ajaWbNO67rx | sudo -u vtest8202 rustic -r rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website backup --stdin-filename vB8ajaWbNO67rx.sql - --password "" --json 2>/dev/null

See the output below, looks like something really strange is going on in terms of path inside rustic, please note that this is a merged snapshot, so it contains other files as well. I see a stdin file at the end.

"home" 
"home/backup" 
"home/backup/v2testbackup.oso5ecswoq.cyberpanel.website" 
"home/backup/v2testbackup.oso5ecswoq.cyberpanel.website/config.json" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba/data" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba/data/30" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba/data/30/30e71493aa4fec41ef15e035c796cd997bf06114723fac8d2eef82626af9f6d0" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba/data/c8" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba/data/c8/c8d25fcb7ad99115f9a19db3f6b978757ebdc810ca9d4574f387ded15c4b5882" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba/index" 

"home/v2testbackup.oso5ecswoq.cyberpanel.website/public_html/xmlrpc.php" 
"stdin" 

Update: on another non-ubuntu system, everything work as expected

"home/cyberpanel.net/public_html/wp-includes/wp-diff.php" 
"home/cyberpanel.net/public_html/wp-links-opml.php" 
"home/cyberpanel.net/public_html/wp-load.php" 
"home/cyberpanel.net/public_html/wp-login.php" 
"home/cyberpanel.net/public_html/wp-mail.php" 
"home/cyberpanel.net/public_html/wp-settings.php" 
"home/cyberpanel.net/public_html/wp-trackback.php" 
"home/cyberpanel.net/public_html/xmlrpc.php" 
"home/cyberpanel.net/usman.jVceqzQ175.cyberpanel.website" 
"home/cyberpanel.net/usman.jVceqzQ175.cyberpanel.website/index.html" 
"qf4fza8IocVVqU.sql" 
"yKtbkp1THRlmBk.sql"

Update 2: Rather then picking the name from stdin, it is just calling it stdin

root@v2tests:~# sudo -u vtest8202 rustic -r rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website ls 03725241
using no config file (./rustic.toml doesn't exist)
enter repository password: 
[INFO] repository rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website: password is correct.
[INFO] using cache at /home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba
[00:00:00]                                β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ          0/0                      [INFO] getting snapshot...
[00:00:00] reading index...               β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ         12/12                     

"stdin"

This is a bug, see backup from stdin: --stdin-filename is not used in path Β· Issue #575 Β· rustic-rs/rustic Β· GitHub.
I’ll fix it ASAP.

Looks like does not happen on alma linux but on ubuntu only? Because on almalinux I see correct files.

Working together for open source. Very good! Looking forward to this major update. :slight_smile:

1 Like

The fix is included the the just-released version 0.5.1

1 Like

noted, thanks alot