CyberPanel Backups v2 Proposal

Update 13th Feb: Consider Linode for S3 Storage as option.

The majority of my sites aren’t Wordpress, unfortunately which means I’m stuck with just SFTP or Local backups. I have tried everything but SFTP just doesn’t work, same issue as these posts 1, 2, 3.

Because of this I don’t use CP backups. My current backup method is scheduling Chronosync on my Mac to SFTP into my server and syncing folders, and then on my Mac, creating copies/retention of these offsite copies. It’s not ideal, and I don’t have a good way of backing up databases, and mail.
Really hoping we can use Backblaze in V2.

Backups v2 will be at the website level, which will include everything.

1 Like

Do you think prepare possibility for set backup for Admin? I think about one place where can you set type of backup for all sites where you can choose how often make incentermal backup and just run.
Because when you have a lot of website is take a lot of time enable backup site by site.

I am sorry but I don’t understand the problem?

Probably make one click backup backup all sites and mails with out need adding every site separately for backup job.

1 Like

The best options are:

Amazon S3, Google Cloud and Wasabi

The first 2 are the largest on the market with infinite options and the latter is the best cost-benefit option and it also has all the necessary options.

In addition, Wasabi does not charge for the input band or output and starts with valuable 1TB/6$

Without any doubt, these 3 are the best, I have tested all the others and have not had good experiences. :slight_smile:

That is something we’ve planned already.

Google cloud or drive? What specifically in Google cloud?

Google Cloud Storage (Google Cloud Platform)…

I migrated all my backups and projects to GCP and i’m currently enjoying it more than aws.

I also use wasabi for long-term backups, due to the incredible 1TB capacity without access limitation. :slight_smile:

I believe they are the best options, especially gcp. But of course, that’s just my opinion based on the experiences I’ve had, but also analyzing other opinions on the Internet, I realized that they are in fact the best, at least these days. :slight_smile:

I see, thanks for the input.

1 Like

The current SFTP always defaults to user “root” so fixing that to allow any user name would be great.

Because of the SFTP user name fault, my current backup strategy is daily backup to local folder, delete older than 5 days and then sync using rclone to the preferred backend. That is currently SFTP with added encryption by rclone so backups are secure regardless of where they are stored.

There was some discussion a while back to integrate rclone into the CP backup giving the massive backend compatibility of rclone. Usmannasir, is that still an option or is it complicating things too much?

This is something I am 100% going to take into considerations. Will allow non-root or virtually any user to use for sftp backups.

We can use rclone for remote destinations, I am still working on the core of backups as of now (like how to generate the actual back file)

3 Likes

Fair enough, one step at a time. The non root user would be a real addition for the moment.

I can’t say it enough how much I appreciate what you and the team have created in Cyberpanel and the time you have spent on it. The odd bug can be worked round but the core is a solid CP that ticks more boxes than almost all CP’s.

1 Like

I allready proposed and addressed some issues previously.

Email notifications on failed backups
Current solution fails silently, which happened a few times. Mostly my fault, but it scares me that backups can fail for days or even weeks without me knowing.

Link backups schedules to packages
It would be nice to be able to select default backup schedules for packages and having new websites added to the schedule automatically.
This would make it possible to offer daily backups for high tier clients and weekly for low tier clients easily. And you won’t have to worry about forgetting to add a website to the backups.

Cleanup temporary backups files
Currently the backup creates random files and doesn’t cleanup after itself. The files are small, but a million small files… I’m sure you get the point.

Email backups in separate schedule
I’m not sure if email is included in the backup now. But it would be nice to be able to separate it from the rest.

Have only one backup menu in the Cyberpanel backend
There are multiple locations for different backups now. Would be nice to have them in one place

Incremental backups on Wordpress backups
I couldn’t find it in the current solution, so I’m just adding it here. I use the Wordpress backups on S3 Wasabi. I backup the DB hourly and files daily. Having an incremental option for files would drastically decrease my backup storage size.

Backup retention on S3
This is not working out of the box but would be a requirement if you ask me.

That’s what I’ve cooked up. What do you think?

Thanks for a detailed post.

I’ve updated the main thread and picked some points from your thread.

However rather then backing up specific directories, how about by default backing up all and ability to exclude some directories?

PS: Do check the original post too.

Link backups schedules to packages

We can attach backup schedules to packages, but how can you configure the run time from there? there is no such option.

But for any website under a certain package will be automatically added to schedule. But I personally believe it will confuse lots of users.

Cleanup temporary backups files

Definately, all temporary files will be removed after final zip is prepared.

Email backups in separate schedule

This will be included

Incremental backups on Wordpress backups

Backups v2 will not be incremental, unless there is a tool you can suggest apart from restic?

Incremental backups are essential - it would be a waste of time to develop Backups v2 without a solid incremental mechanism.

I previously shared a lot of ideas and information regarding cyberpanel’s implementation of incremental backups in this Github issue. RClone for Incremental Backups · Issue #831 · usmannasir/cyberpanel · GitHub.

In short, restic is completely unsuitable for this task and should have been removed long ago. However, there’s a fork/companion of it called Rustic (written in rust) that satisfies all of the concerns I shared there, has compression and more.

I would expect that cyberpanel backups v2 would incorporate all of this, along with using rclone (which is already integrated into Rustic) to allow for people to use pretty much any backend (cloud storage, ftp, etc…) that they want.

The Rustic Dev is actively developing it and is very responsive to feedback, so I’m sure would be thrilled to work together to make something great and bring a higher profile to both tools.

1 Like

Hi, nick

Glad to see you back. I will be honest with you, developing the incremnting files backup is kind of out of the scope of our development unless there is already some tool that is doing this and we can incorporate it in CyberPanel such as rustic.

I am just afraid that what if down the line even if rustic developer stops working on this tool and we will then end up wasting lots of our time. With that being said I am going to explore this option for sure and see if we can take care of incremental backups. The important thing is that I need to ensure that restore of the backups is as easy as possible be it normal or incremental backups.

I will study and revert to you as well.

thanks

2 Likes