CyberPanel Community

CyberPanel Backups v2 Proposal

RM
Rana Muhammad Usman Nasir #1

Hello everyone and I hope everyone is having a good new year so far

As per many requests in the group and our community, we’ve planned to do backups v2 for CyberPanel. They will be really robust backups in terms of features and backup destinations.

So we are looking for suggestions, I will lay out the basic plan below.

1. Option to allow backups

For example do you wan to back website data, databases, emails etc?

2. Option to exclude directories from backups backups

Allow users to exclude directories from backup.

3. Multiple backup frequencies

Daily, weekly, monthly?

4. Multiple destinations?

Google drive, sftp, (FTP Not recommended) or more?

Update 12th Feb: Setting up custom paths for Backups generation activity.
Update 13th Feb: Consider Linode for S3 Storage as option.

Update on 17th Feb:

Some key takeaways that I am considering from a post by user here.

  1. option to exclude subdomains or staging websites: Providing the ability to selectively backup specific domains or exclude subdomains and staging websites will offer greater control and flexibility for website owners.
  2. option for selecting specific directories: Implementing an option for backing up specific directories will simplify the management and restoration of backups.
  3. Lack of hourly database schedule: Introducing a more frequent backup schedule for databases will increase the reliability and efficiency of the backup system.

Update 2: on 17th Feb:

Take aways from this comment

Email notifications on failed backups
Current solution fails silently, which happened a few times. Mostly my fault, but it scares me that backups can fail for days or even weeks without me knowing.

Link backups schedules to packages
It would be nice to be able to select default backup schedules for packages and having new websites added to the schedule automatically.
This would make it possible to offer daily backups for high tier clients and weekly for low tier clients easily. And you won’t have to worry about forgetting to add a website to the backups.

to be continued…

143 replies
St
Steve #3

This is exciting news! I’ve struggled with the CP Backups interface, and I’m very keen to assist in making V2 something that’s easier to set up and use, and provides more peace of mind.

I think it’s worth looking at how other panels do it, to get ideas, create a discussion and ultimately formulate a plan based on what most people’s needs are. There are also some good backup workflows found in various WP Plugins, so feel free to share those.

Here’s how WHM handles Backups and Restoration.

WHM Backup Settings

WHM Backup Destinations
whm-backupdestinations

WHM Backup Restoration

I personally think WHM’s approach is also a bit convoluted and confusing, and can be improved upon, but I do really like the interface for Scheduling and Retention. I’m also drooling over all those backup destinations. Backblaze B2 would be one I’d personally utilise.

RM
Rana Muhammad Usman Nasir #5

It looks like cPanel backups kind of complex to configure for non-technical users. (I am not talking about their functionality, but how they are configured)

Dr
Dreamer #6

Function to use any S3 compatible location would be good. Is this new backup be paid addon or apply default incremental backup system too?

Br
Brad #7

I am starting on a very tight budget and can’t afford Amazon S3.

I have found out that Linode is now offering a new storage offering that is compatible with S3 compatible, and is very cheap for a good monthly transfer amount.
I’ve also used the free level of Google Drive and what comes with the free is very workable unless you have a huge website and a huge database for Word press.
(Ps I found out what I couldn’t repond to anything or click on anything in here. Apparently these pages don’t like Google Chrome, what I switched to Firefox it worked fine.

Dr
Dreamer #8

How much you have data to backup? I you look cheapest solution take look on wasabi and backblaze as long we would get support for those location’s.

RM
Rana Muhammad Usman Nasir #10

Wasabi, Backblaze are already currently supported in WordPress Manager. (Amazon s3 too)

Dr
Dreamer #11

Any plan support S3 compatible storages on free incremental backup option or that’s not developed anymore?

RM
Rana Muhammad Usman Nasir #12

The problem with current incremental backups are the password needed to unlock the backup repo (we use restic as backend for incremental backups)

And almost 90% of people loose their password, normally its present in the doc root of the site.

Br
Brad #13

“If Linode is S3 supported it wont be really hard to implement them.”
Any documentation forthcoming when the upgrade comes in with the new implementation?

I’ve been a network admin and then later a web admin I know how to get around a lot of stuff and play until I find my answer but sometimes I get stumped.

I used to use CPanel and WHCM but they got too expensive a while back.
I switched to Plesk and finally got most of what I needed figured out and got tired of them raising their licensing prices and then they merged with the company that owns cPanel then they started nickle and diming me for the extensions which used to be reasonably priced.

When I watched a tutorial video from Idea spot talking about how to install cloud panel and it was open sourced and typically free… It does just about everything that Plesk does without charging you an arm and a leg for modules or extensions.

RM
Rana Muhammad Usman Nasir #14

I’ve just started writing the code for Backups v2, they will be at website level including and taking care of everything including data, emails, wordpress sites, emails.

Right now just the begining can take 3-4 months from today and then the documentation.

However the current solutions we’ve are also doing good.

Is there a significant difference in pricing between backblaze vs linode?

Br
Brad #15

You mean 90% of users don’t put user names, pw’s and urls and titles and vendor name into a spreadsheet?

With the ease of creating spreadsheets in Micrososft Excel, in Windows or Libre Office Calc or WPS spreadsheet, why don’t people put everything in a spreadsheet? geesh - do you gotta hold their hand and wipe their tooshie too?

RM
Rana Muhammad Usman Nasir #16

I would return to this discussion later. However if you try to use incremental backups you may get an idea what I am trying to say.

I will write some code now to lay the foundation for the backups v2.

thanks

Br
Brad #17

to be honest I just chose linode because im familiar with them from using their vps in the past and they’re known for quality and savings.

I’ll be more than happy to look into backblaze since they’re already in the system.
I only responded with linode because it looked like you wanted feedback of what people were interested in usin but I’ll look backblaze.

Dr
Dreamer #18

For addons onetime payment option I was asking before possibility to change IP for subscription, if move other VPS provider. I was told that that’s not possible because some people might misuse that so I decided not to buy that. Any possibility make possible change server IP for that subscription like once year? That way people cant abuse subscription and if I need some reason change VPS provider like in one year I wont waste my money?
Like me I only use email debugger other option not that necessary but wont hurt to have those if get on time deal work…

RM
Rana Muhammad Usman Nasir #19

If you want to only use email debugger you won’t need to buy all add-ons package. Email debugger is only $2.99.

And we’ve added the policy of changing IP within 30 days of purchase, also the life-time may not stay there forever, limited time offer for now.

Dr
Dreamer #20

I have been buying debugger once every every 3 months but anyway would be more than happy to buy life-time deal if in future if update hardware, move dedi etc… can move to new server.

St
Steve #22

The majority of my sites aren’t Wordpress, unfortunately which means I’m stuck with just SFTP or Local backups. I have tried everything but SFTP just doesn’t work, same issue as these posts 1, 2, 3.

Because of this I don’t use CP backups. My current backup method is scheduling Chronosync on my Mac to SFTP into my server and syncing folders, and then on my Mac, creating copies/retention of these offsite copies. It’s not ideal, and I don’t have a good way of backing up databases, and mail.
Really hoping we can use Backblaze in V2.

In
Infosun #24

Do you think prepare possibility for set backup for Admin? I think about one place where can you set type of backup for all sites where you can choose how often make incentermal backup and just run.
Because when you have a lot of website is take a lot of time enable backup site by site.

Dr
Dreamer #26

Probably make one click backup backup all sites and mails with out need adding every site separately for backup job.

JA
Joabe Arruda #27

The best options are:

Amazon S3, Google Cloud and Wasabi

The first 2 are the largest on the market with infinite options and the latter is the best cost-benefit option and it also has all the necessary options.

In addition, Wasabi does not charge for the input band or output and starts with valuable 1TB/6$

Without any doubt, these 3 are the best, I have tested all the others and have not had good experiences. :slight_smile:

JA
Joabe Arruda #30

Google Cloud Storage (Google Cloud Platform)…

I migrated all my backups and projects to GCP and i’m currently enjoying it more than aws.

I also use wasabi for long-term backups, due to the incredible 1TB capacity without access limitation. :slight_smile:

I believe they are the best options, especially gcp. But of course, that’s just my opinion based on the experiences I’ve had, but also analyzing other opinions on the Internet, I realized that they are in fact the best, at least these days. :slight_smile:

he
hexitnz #32

The current SFTP always defaults to user “root” so fixing that to allow any user name would be great.

Because of the SFTP user name fault, my current backup strategy is daily backup to local folder, delete older than 5 days and then sync using rclone to the preferred backend. That is currently SFTP with added encryption by rclone so backups are secure regardless of where they are stored.

There was some discussion a while back to integrate rclone into the CP backup giving the massive backend compatibility of rclone. Usmannasir, is that still an option or is it complicating things too much?

RM
Rana Muhammad Usman Nasir #33

This is something I am 100% going to take into considerations. Will allow non-root or virtually any user to use for sftp backups.

We can use rclone for remote destinations, I am still working on the core of backups as of now (like how to generate the actual back file)

he
hexitnz #34

Fair enough, one step at a time. The non root user would be a real addition for the moment.

I can’t say it enough how much I appreciate what you and the team have created in Cyberpanel and the time you have spent on it. The odd bug can be worked round but the core is a solid CP that ticks more boxes than almost all CP’s.

IS
Imrul Sadman #35

I allready proposed and addressed some issues previously.

xm
xmedia #36

Email notifications on failed backups
Current solution fails silently, which happened a few times. Mostly my fault, but it scares me that backups can fail for days or even weeks without me knowing.

Link backups schedules to packages
It would be nice to be able to select default backup schedules for packages and having new websites added to the schedule automatically.
This would make it possible to offer daily backups for high tier clients and weekly for low tier clients easily. And you won’t have to worry about forgetting to add a website to the backups.

Cleanup temporary backups files
Currently the backup creates random files and doesn’t cleanup after itself. The files are small, but a million small files… I’m sure you get the point.

Email backups in separate schedule
I’m not sure if email is included in the backup now. But it would be nice to be able to separate it from the rest.

Have only one backup menu in the Cyberpanel backend
There are multiple locations for different backups now. Would be nice to have them in one place

Incremental backups on Wordpress backups
I couldn’t find it in the current solution, so I’m just adding it here. I use the Wordpress backups on S3 Wasabi. I backup the DB hourly and files daily. Having an incremental option for files would drastically decrease my backup storage size.

Backup retention on S3
This is not working out of the box but would be a requirement if you ask me.

That’s what I’ve cooked up. What do you think?

RM
Rana Muhammad Usman Nasir #37

Thanks for a detailed post.

I’ve updated the main thread and picked some points from your thread.

However rather then backing up specific directories, how about by default backing up all and ability to exclude some directories?

PS: Do check the original post too.

RM
Rana Muhammad Usman Nasir #38

Link backups schedules to packages

We can attach backup schedules to packages, but how can you configure the run time from there? there is no such option.

But for any website under a certain package will be automatically added to schedule. But I personally believe it will confuse lots of users.

Cleanup temporary backups files

Definately, all temporary files will be removed after final zip is prepared.

Email backups in separate schedule

This will be included

Incremental backups on Wordpress backups

Backups v2 will not be incremental, unless there is a tool you can suggest apart from restic?

Ni
Nick #39

Incremental backups are essential - it would be a waste of time to develop Backups v2 without a solid incremental mechanism.

I previously shared a lot of ideas and information regarding cyberpanel’s implementation of incremental backups in this Github issue. RClone for Incremental Backups · Issue #831 · usmannasir/cyberpanel · GitHub.

In short, restic is completely unsuitable for this task and should have been removed long ago. However, there’s a fork/companion of it called Rustic (written in rust) that satisfies all of the concerns I shared there, has compression and more.

I would expect that cyberpanel backups v2 would incorporate all of this, along with using rclone (which is already integrated into Rustic) to allow for people to use pretty much any backend (cloud storage, ftp, etc…) that they want.

The Rustic Dev is actively developing it and is very responsive to feedback, so I’m sure would be thrilled to work together to make something great and bring a higher profile to both tools.

RM
Rana Muhammad Usman Nasir #40

Hi, nick

Glad to see you back. I will be honest with you, developing the incremnting files backup is kind of out of the scope of our development unless there is already some tool that is doing this and we can incorporate it in CyberPanel such as rustic.

I am just afraid that what if down the line even if rustic developer stops working on this tool and we will then end up wasting lots of our time. With that being said I am going to explore this option for sure and see if we can take care of incremental backups. The important thing is that I need to ensure that restore of the backups is as easy as possible be it normal or incremental backups.

I will study and revert to you as well.

thanks

Ni
Nick #41

I appreciate your concerns, though they apply to Restic and any other tool that is used by Cyberpanel.

For what it’s worth, I found both Restic and Rustic to be very easy to use, with Rustic being better because it handles incremental backups properly, is faster, has a good resume mechanism etc…

I really don’t think it would be a very difficult job to integrate into Cyberpanel - I managed to build most of the CLI stuff in the PRs that I had submitted. It was just the HTML interface (Django etc…?) that I couldn’t figure out.

I hope you’ll be able to make it work - it’s a primary request for many people!

RM
Rana Muhammad Usman Nasir #42

Honestly nick I was just checking rustic I was not able to find doc links on their repo.

Plus the main concern is how to restore those rustic backups if customer loose complete access to server as these are password protected repos plus it is still really slow for large files.

These were the biggest constraints and still feels the same constraint applies.

RM
Rana Muhammad Usman Nasir #43

One more thing I like to add is that for backups on the active server (server which is not yet crashed), rustic/restic works.

But incase server crashes from some reason then rustic/restic wont work, because most people won’t save the password that is required to unlock repo, thus the remote repos become useless.

Another important bottleneck is still the speed. :frowning:

Ni
Nick #44

I believe you can use restic and Rustic without any sort of password. But that’s a non-argument anyway - the same thing applies to mysql and cyberpanel itself.

And so long as you have access to the Rustic backup repo, you can restore it to anywhere, such as a new server.

I don’t see why it would be really slow for large files - it should be faster for them than smaller ones. Plus you can configure different compression levels, or turn it off altogether. Again, in my testing long ago (and it surely has improved since then), creating and restore backups was 2-3x faster with the compression on than with it off.

Also, incremental backups are, obviously, far faster to create and restore than a full backup and take up incomparably less storage space, which allows for setting a schedule like every 30 minutes, which is more secure/reliable than infrequent large backups.

I’m quite certain that any of your concerns could be easily alleviated with some combination of reading the cli docs/help command, experimenting, and just asking the dev for help.

he
hexitnz #45

For me, the current backup restore works very well. I use it for moving server’s as well as the occasional restore. The simplicity of it means I can rely on it working quickly and efficiently without passwords keys etc. Sometimes simplicity is good.

Picking the location of the restore file would be very useful rather than having to use the set /home/backup.

Individual restore options, database, email, files etc would be very useful.

RM
Rana Muhammad Usman Nasir #46

For rustic which docs are you using? As far as I am aware can’t create repo without password, correct me If I am wrong?

Ni
Nick #47

I haven’t used Rustic in 6 months, but I recall being able to type something like rustic --help to see all docs. And, again, the dev is very responsive so you could ask questions in the Github repo if anything isn’t clear. I think he’d be particularly eager to respond and help if you said you were trying to integrate it into an open source control panel.

RM
Rana Muhammad Usman Nasir #48

I think it would be better to get in touch as we are in really initial stage now. Let see how it goes, but if it works the same as restic, my hand would be tied as to avoid using it.

Ni
Nick #49

Here’s how to access the docs/cli options

What do you mean by “but if it works the same as restic, my hand would be tied as to avoid using it”?

As for creating a repo without a password, I’m not sure. I suspect it is possible, but maybe not. However I’m quite confident that the dev would quickly add such a possibility if it were requested.

But, again, it’s a non-issue to me… There’s already passwords for accessing a server, using cyberpanel, accessing mysql, and for using any other service in existence. If someone can’t figure out passwords, that’s their problem, not cyberpanel. But, again, I’m quite sure it would be possible to set up backups without any passwords) other than the pw required to access whatever rclone-compatible data store is being used

Moreover, restic - which is already used by cyberpanel - has the same password mechanism, which im pretty sure you’ve just been storing in a plaintext file somewhere on the server, just as you do for the mysql password.

Edit: I’ve created issues in Rustic to request being able to create repos without a password and without encryption. We’ll see what the dev says.

su
subzero06 #50

I also use Linode as host and they do offer S3 compatible storage - would be nice it we could use any S3 compatible storage.

The more options the users has, the better, that way we are not locked to whatever is provided. You could even make it a “paid” addon feature.

With failed email notifications as well - imagine you think your stuff is backing up and month later you check that an important backup has failed and you are not even aware. That would drive clients to not use your product.

Oc
Oceanwatcher #51

First of all:

Website backup - if anyone of you have not tried Akeeba Backup (Wordpress or Joomla) you should check it and see how things are done there.
In the pro version, you can choose to backup to a variety of external storage. You can schedule however you like - and decide if you want to do the whole website incl database or just the database+++

I would also like to suggest Nextcloud as a possible target for storage/backup.

he
hexitnz #52

As Nextcloud does WebDAV, that would be a good general option to add.

Ni
Nick #53

Rustic Dev confirmed it can be used without password - just enter a blank string. Allow empty passwords · Issue #433 · rustic-rs/rustic · GitHub

Again, it really seems like the ideal tool for the job. It would work well with the existing restic mechanism, while resolving all the problems with restic. And, as shown here, the dev is very responsive (replied in well under an hour) and eager for collaboration.

RM
Rana Muhammad Usman Nasir #54

Dear Nick,

I hope this message finds you well. I wanted to share some important information with you about the command currently being used by CyberPanel to create restic backups.

The command in use is as follows:

bash

restic -r /home/cyberpanel.net/incbackup backup /home/cyberpanel.net/meta.xml --password-file /home/cyberpanel.net/cyberpanel.net --exclude /home/cyberpanel.net/incbackup --exclude /home/cyberpanel.net/backup

I’d like to draw your attention to the --password-file /home/cyberpanel.net/cyberpanel.net argument which is used both during backup creation and restoration. This password is essential, even for remote backups such as AWS.

It is worth noting that for SFTP and other normal backups, passwords are not required to restore or access backup files. However, in the case of restoring incremental backups, it is crucial to have this password.

In the event that a user is unaware of the password file and their server crashes, they may be unable to restore their backups. To address this issue, we can take one of two approaches: firstly, we can educate the user about the importance of the password and encourage them to save it locally. Secondly, we can explore alternative backup tools that do not require a password.

It’s important to note that the restic/rustic password is distinct from other passwords such as SSH or MySQL. If a server crashes, the backups become the most critical aspect of recovery, and without the restic/rustic password, the backups are useless.

However, there may be workarounds available, such as using different backup tools. Additionally, we can take proactive steps by ensuring users are aware of the importance of saving passwords in a safe location.

Please let me know if you have any questions or concerns. Thank you for your time.

Best regards,

JA
Joabe Arruda #56

I’m lieked the way things go here. :sunglasses::rocket:

I only know one thing, the cyberpanel will be one of the main ones very soon. Even a giant like hostinger/weblink has already joined. You guys have a great job. :clap::clap:

Ni
Nick #57

Thanks for the nice and detailed response! My point about passwords was that we already use and manage numerous passwords per server, and likely dozens or hundreds through life. So a backup password really isn’t a big deal.

Anyway, Rustic doesn’t require one like Restic does, so there’s no concern. It’s just one of many ways in which it is a far superior tool, so I’m glad that you’ll give it some serious consideration now! Let me know if you have any other questions.

xm
xmedia #58

Perhaps an option in the backup schedule to auto-add websites from a certain package. At this moment you need to hand-pick the sites to backup. Maybe add a toggle there to select the package to use as site-list.

That way you keep the backup settings at the backup settings. Would that make sense?

RM
Rana Muhammad Usman Nasir #59

OK Nick, I was just out with the team during the week-end.

I am gonna do some boiler plate code to see how we can get rustic going.

We will start with the installation of it, then backups and see how we can make use of it.

By the way thanks for the input in this regards so far.

PS: Attaching the picture of where our team spent the weekend.

Ni
Nick #60

Looks like a beautiful place! I hope you guys had a great time.

Let me know if you have any questions while testing Rustic!

Su
Sunman #61

I think OneDrive support is required. I have an Office 365 subscription with 1TB of OneDrive cloud, which I can’t use for backups.

he
hexitnz #62

Suman, until the backup has been sorted I recommend rclone for almost any backend. I run a cron script after CP does a backup to a local folder. The script uploads the just created backup to the remote and then deletes the local copy. Actually I keep 3 days backup locally and delete older than that.

It is two step process but it does allow you to use almost all the storage options available.

Ni
Nick #63

@Sunman @hexitnz if they end up implementing the tool (Rustic) that I recommended, you’ll be able to do compressed incremental backups to onedrive and dozens of other services, all from the Cyberpanel control panel.

he
hexitnz #64

Ah I see from the github page that restic can use rclone for it’s backend.

Su
Sunman #66

Yes, rclone is a good option. Thank you!

Ni
Nick #68

Looks like the dev resolved it in a few minutes! Like I’ve said, they’re very responsive and eager to receive feedback and contributions (whereas Restic is semi-abandonned).

I look forward to hearing how your early tests go - I expect you’ll be able to integrate it with the existing cyberpanel incremental backup mechanism very easily.

Some things to note:

  • I believe that in my testing, the default compression level (zstd 3, I think) provides the best balance between speed and size. It is obviously a bit slower when the backup is to a local drive, but considerably faster than uncompressed when sent remotely over the network.
  • And, of course, incremental is infinitely faster and smaller than full backups.

And, just a reminder, I left a lot of detailed notes about issues and desired improvements for the CP backup system here RClone for Incremental Backups · Issue #831 · usmannasir/cyberpanel · GitHub

RM
Rana Muhammad Usman Nasir #69

Yes they do seems really responsive. However restic documentation is really great compared to rustic.

RM
Rana Muhammad Usman Nasir #70

Also we won’t touch our current implemtation of Inc Backups.

Those had some security concerns due to which we were not able to bring sftp back to them, this time we want to ensure that there are no security concerns and things are done right from the start.

Ni
Nick #71

For the most part you can use Rustic’s documentation for Restic. But I found it to be easy to use, through the combination of the CLI docs and responsive Dev. It is not a complicated tool to use.

And, sorry to say, but it’s a bit odd for cyberpanel to complain about someone else’s documentation, when yours is basically nonexistent (despite the hundreds of hours I spent, with minimal collaboration from others, to create a system around it…). Yet we use cyberpanel because it’s a good tool. As is rustic.

Anyway, glad to hear you’re looking to do this well. I’m sure you’ll be able to borrow some of the code from the previous mechanism.

RM
Rana Muhammad Usman Nasir #72

Nick I would agree that our docs may be scattered along various location, however, we’ve youtube tutorials, actual docs and blog section.

But for rustic I am not able to find any documentation at all, except that its identical to restic.

On the other hand if you go into restic site the installation process is really easy.

PS: I really appreciate your efforts to-date regarding moving our forums to discource and arranging the docs.

thanks

he
hexitnz #73

Can the current simple backup to local schedule be kept as well? The simplicity is useful and great for migrating sites to other servers.

Ni
Nick #75

Again, your complaints about Rustic’s docs are misplaced and just seems like a poor excuse for why you shouldn’t explore this option.

Rustic is a new tool and the dev evidently has not had time to create docs. But the Restic docs apply for the most part, and the CLI docs and extremely responsive dev should be more than enough to figure out what you need.

You’re surely 100x the developer that I am, yet I figured out Rustic in a few hours. It is not complicated AT ALL .

The difficult part will be on the Cyberpanel side - integrating it into the panel gui, generating the Sql and filesystem backups, etc… I have little to contribute to that, other than the detailed suggestions that I laid out in that Github issue that I’ve linked to a couple times.

Again, If people applied the same logic to cyberpanel that you are to Rustic, no one would use it because the docs are a complete disaster - out of date, disorganized, incomplete, poorly written, etc… Yet, we persist with Cyberpanel because it is a good tool. As is rustic.

Still, there’s a huge amount of people who would never touch Cyberpanel because of the terrible docs - which is simply inexcusable for a project of this scale and longevity. So, I hope you’ll get your team to start actually writing, editing, and organizing the Cyberpanel docs, and forums in general, rather than keep wasting your time and resources writing completely superficial and largely irrelevant blog posts.

Fixing the docs would be actually useful to users, and would finally make my hard work worthwhile - as it stands, I feel that my effort was a complete waste of time, so I eventually just gave up out of futility. I’m only back now because I see that you’re finally paying attention to backups, which I already put a lot of work into

RM
Rana Muhammad Usman Nasir #76

There is no excuse man, if you check v2.3.4-dev branch we have already started writing some code.

But I stand my words, our docs are not as bad as the rustic ones plus it is not 100% restic compatible, for example the --exclude does not work. But we are working on it, and we will know in couple of days if this is something we will be using in a long run

thanks

Ni
Nick #77

Rustic is an immensely more simple tool than CyberPanel thus docs are far less necessary. Again, I - someone who really is not particularly capable with any of this - figured it out in a few hours. It is also a considerably younger tool with essentially one contributor, so actually has an excuse for not having good docs yet - none of that is the case for CyberPanel, especially when volunteer-efforts were squandered.

But here’s a PR that is already in progress for Rustic Docs WIP: Add docu by aawsome · Pull Request #438 · rustic-rs/rustic (github.com). The dev would greatly appreciate any edits/suggestions/contributions to it.

Yes, it is not 100% Restic compatible with regards to the specific commands/api. But, that is largely irrelevant and another lame excuse. What is important is that it is compatible with the Restic Repo format such that you could switch from Restic to Rustic and maintain continuity with the same repos.

The exclude parameters are in the CLI docs (which are dead-simple to access via ./rustic backup -h and they seem quite flexible.

And they’re also listed here in the Docs PR with some examples: WIP: Add docu by aawsome · Pull Request #438 · rustic-rs/rustic (github.com).

Again, if that was a sticking point for you, the dev would have been happy to explain it if you opened an issue for it.

And, just to make it clear/easy, you can type -h after any of the commands to see the full documentation. Its quite extensive and, I think, clear. Here’s what shows up for ./rustic -h. Adding any of the subcommands (backup, init, etc…) will show their respective docs.

USAGE:
    rustic [OPTIONS] <SUBCOMMAND>

OPTIONS:
    -h, --help                        Print help information
    -P, --config-profile <PROFILE>    Config profile to use. This parses the file `<PROFILE>.toml`
                                      in the config directory [default: rustic]
    -V, --version                     Print version information

GLOBAL OPTIONS:
        --log-file <LOGFILE>       Write log messages to the given file instead of printing them.
                                   Note: warnings and errors are still additionally printed unless
                                   they are ignored by --log-level [env: RUSTIC_LOG_FILE=]
        --log-level <LOG_LEVEL>    Use this log level [default: info] [env: RUSTIC_LOG_LEVEL=]

REPOSITORY OPTIONS:
        --cache-dir <CACHE_DIR>
            Use this dir as cache dir instead of the standard cache dir [env: RUSTIC_CACHE_DIR=]

        --no-cache
            Don't use a cache [env: RUSTIC_NO_CACHE=]

    -p, --password-file <PASSWORD_FILE>
            File to read the password from [env: RUSTIC_PASSWORD_FILE=]

        --password <PASSWORD>
            Password of the repository - WARNING: Using --password can reveal the password in the
            process list! [env: RUSTIC_PASSWORD=]

        --password-command <PASSWORD_COMMAND>
            Command to read the password from [env: RUSTIC_PASSWORD_COMMAND=]

    -r, --repository <REPOSITORY>
            Repository to use [env: RUSTIC_REPOSITORY=]

        --repo-hot <REPO_HOT>
            Repository to use as hot storage [env: RUSTIC_REPO_HOT=]

        --warm-up
            Warm up needed data pack files by only requesting them without processing

        --warm-up-command <WARM_UP_COMMAND>
            Warm up needed data pack files by running the command with %id replaced by pack id

        --warm-up-wait <DURATION>
            Duration (e.g. 10m) to wait after warm up

SUBCOMMANDS:
    backup         Backup to the repository
    cat            Show raw data of repository files and blobs
    check          Check the repository
    completions    Generate shell completions
    config         Change the repository configuration
    copy           Copy snapshots to another repository
    diff           Compare two snapshots/paths
    forget         Remove snapshots from the repository
    help           Print this message or the help of the given subcommand(s)
    init           Initialize a new repository
    key            Manage keys
    list           List repository files
    ls             List file contents of a snapshot
    prune          Remove unused data or repack repository pack files
    repair         Restore a snapshot/path
    repoinfo       Show general information about the repository
    restore        Restore a snapshot/path
    self-update    Update to the latest rustic release
    snapshots      Show a detailed overview of the snapshots within the repository
    tag            Change tags of snapshots
RM
Rana Muhammad Usman Nasir #78

I think you are not following our dev branch where we are actively working on rustic. Commits · usmannasir/cyberpanel · GitHub

Once again if at any point I feel that rustic is not the tool we need (I know you like it very much) I will not be going to use it.

PS: Not everyone is like you who would like to go to rustic help section on the CLI. When we implement some tool eventually some day users may need to figure out what is going on behind the scene.

I am not going to further waste time on discussion what is good and what is not while we are already working with rustic, but as I said above if I feel at some point that this is something we don’t need (by which I mean: it wont work in our usecase) we will not use it.

aa
aawsome #82

rustic author here.

Yes, if you want to limit cpu/io/network bankdwidth, all restic/rclone docs also apply to rustic.

@usmannasir I had a quick look at your changes in Backupsv2.py and have a few comments (feel free to ignore what ever you think is not helpful):

  • rustic also supports a config file. If you already create a config file for rclone, you can as well use one for rustic and e.g. put repository and password in it. Then your rustic calls will be much cleaner.
  • as @nick.chomey alredy mentioned, there are the --glob options which can be used to exclude patterns. However, the pattern must be inverted when comparing to a --exclude option, i.e. use !file in order to exclude file. I realize this is somehow strange, so I might add some --exclude options as well - if this is important for you, please open an issues for rustic.
  • You are using --parent to identify some snapshots. There are two other possibilities which I think would cover this use case better, namely a label (optionally set per snapshot) or tags (you can have multiple per snapshot). You can set these by using rustic backup --label <LABEL> or rustic backup --tag <TAG> and can use them to filter your snapshots list: rustic snapshots --filter-label <LABEL> --filter-tags <TAG>
RM
Rana Muhammad Usman Nasir #84

Welcome @aawsome

  1. Yes recently I just wrote the code for using the rclone config files which made overall code much cleaner.
  2. noted
  3. I am using parent, because as I was discussing with @nick.chomey in facebook that we need to make this repo independant of CyberPanel. So if CyberPanel server is completely lost, we’ve a config file attached to each thing (databases, db, emails) so that site can be rebuilt with it. I looked into label but had some issues.
aa
aawsome #87

I just checked it out. I’m backing up using

mysqldump -u <USER> -p <PASS> --allow-keywords --add-drop-table --complete-insert --quote-names --skip-comments <DB> \
| rustic backup --stdin-filename <DB>.dump -

As mentioned you should try out if this suffices to get an unchanged dump for an unchanged DB. If not, you might have to add additional --skip-* options.

Ni
Nick #88

Thanks very much. As it turns out, I’ve now decided to set up some cron backups of some web applications not managed by/associated with CyberPanel. I’ll give this a try and report back if I find anything different!

Ni
Nick #89

@aawsome

I’ve done some testing for the command you provided.

It would appear that the crucial flag is --skip-comments if you want to avoid backing up anything when there is no change.

Also, --add-drop-table and --quote-names are already set to TRUE by default so don’t need to be specified.

What purpose do you have for using --allow-keywords and --complete-insert? (reading the docs, they do appear like they probably should be used together)

Finally, it looks like if you make a minor change to the db (one word, one row, etc…), it adds 444.4 kiB (raw: 2.5 MiB) to the repo. That’s for a database that was originally 11.8 MiB (raw: 46.9 MiB). Seems a bit excessive, but it is what it is and MUCH better than the full thing. Though, I presume that if you had a 100GB DB, it would still add the same 444kiB, which would be nothing.

Thanks very much for all your help and hard work with Rustic!

I hope this helps @usmannasir.

RM
Rana Muhammad Usman Nasir #90

BTW, how would you structure this command, when one input is from stdin and other is from the file

mysqldump -u <USER> -p <PASS> --allow-keywords --add-drop-table --complete-insert --quote-names --skip-comments <DB> \
| rustic backup --stdin-filename <DB>.dump -

Keeping this in mind: Restore into some specific path · Issue #463 · rustic-rs/rustic · GitHub

Ni
Nick #91

@usmannasir @aawsome

Looks like we have the same question. Here’s my version and attempt. Allow multiple sources within one snapshot by aawsome · Pull Request #458 · rustic-rs/rustic · GitHub

My solution, for now, is to not use stdin. Instead, I’m doing everything from a script that generates the mysqldump file then includes that in the backup, then deletes the file. But it would be nice if stdin could be used alongside other paths.

Also, as mentioned in my comment above, you only need to use the --skip-comments flag. Two of them are already set by default and the other two don’t seem to be necessary.

RM
Rana Muhammad Usman Nasir #92

I was going through different issues created by you on the repo, I would be looking for ideal case, because once this code gets finalized it would be hard to come back and do things differently.

Which is why I am looking for an option that takes a file from stdin and a file path from cli too…

aa
aawsome #93

@usmannasir Backing up from stdin and one or more paths giving a single snapshot is not yet supported.

Currently I’m working on a merge command, see WIP: Merge snapshots by aawsome · Pull Request #411 · rustic-rs/rustic · GitHub which will allow you to do something like this:

rustic backup /path1 /path2 # gives snapshot abc
mysqldump ... | rustic backup - # gives snapshot def
rustic merge abc def # gives final snapshot
rustic forget abd def # remove temporary snapshots - you can add --prune, but there is not much temporary data left to care about

Moreover there is Add multibackup command · Issue #472 · rustic-rs/rustic · GitHub open which (when implemented) might allow to do all the above in one single step using a syntax (to be defined) like

mysqldump ... | rustic multibackup --merge "/path1 /path2" "-" 
aa
aawsome #94

Another solution which is already supported is to generate two snapshots and manually “link” them together. But I wouldn’t use the parent field for it, but use the description instead to link and maybe labels to differentiate between them. It could look like

mysqldump ... | rustic backup --label db-dump - # gives snapshot def..
rustic backup --label paths --description "DB dump: def..." /path1 /path2 # gives final snapshot

Then, you can choose a “paths” snapshots from

rustic snapshots --filter-label paths 

and find the “linked” snapshot by getting the description field and parsing for DB dump: {id}.
But this linking is of course fully outside of what rustic natively supports.

aa
aawsome #95

As the merge command is implemented (but not yet released), you can now use:

rustic backup /path1 /path2 # gives snapshot abc
mysqldump ... | rustic backup - # gives snapshot def
rustic merge abc def --delete # gives final snapshot and deletes the two tmp snapshots

Note that all commands support the --json flag.

RM
Rana Muhammad Usman Nasir #96

OK, I am gonna work with merge now, i think it is going to change whole lots of things.

aa
aawsome #97

One thing to mention: You get fast backups only if you use a parent snapshot. So you might want to keep the original snapshots (or at least one) when backing up local paths. For backups from stdin, this however doesn’t help. Here the input stream has to be processed anyway.

RM
Rana Muhammad Usman Nasir #98

Something wrong?

mysqldump --defaults-extra-file=/home/cyberpanel/.my.cnf -u root --host=localhost --port 3306 --add-drop-table --allow-keywords --complete-insert --quote-names --skip-comments cybe_usman | sudo -u cyber5986  rustic -r rclone:testremote:cyberpanel.net backup --stdin-filename cybe_usman.sql --password "" --json

using no config file (./rustic.toml doesn’t exist)
[INFO] repository rclone:testremote:cyberpanel.net: password is correct.
[INFO] using cache at /home/cyberpanel.net/.cache/rustic/ad21841af5644e9026c2c9a649fc03e9b1aa8fba26c08a13c75d8eb03e8361a8
[WARN] no backup source given.

aa
aawsome #99

You should add the - to backup from stdin: rustic backup --stdin-filename xxx.sql -

aa
aawsome #102

I suppose you mean the rclone command called by rustic, right?
Actually it is not yet possible - except you do a manual hack like rename rclone and use a self-made script with name rclone which calls the renamed original rclone with parameter --config.

Do you need this and plan to use rustic config files? I can easily add an option to specify the rclone-command or rclone options in the rustic config file.

RM
Rana Muhammad Usman Nasir #103

Yes rlcone config file, while doing backup of emails, the user does not have access to rclone backend file.

When the command runs as root it obviously can’t find the rclone backend as rlcone looks into the user directory, thus if we can pass the path of config file manually, it will solve the issue.

aa
aawsome #104

Just saw that rclone seems to support the XDG_CONFIG_HOME and RCLONE_CONFIG env variables.

So, you can either set XDG_CONFIG_HOME which would use $XDG_CONFIG_HOME/rclone/rclone.conf or directly specify the config file in the env variable RCLONE_CONFIG.

Still, I’ll open an issue to let rustic set rclone options.

aa
aawsome #106

@usmannasir You just have to set the RCLONE_CONFIG environment variable. I think this already works with all rustic versions.

aa
aawsome #108

Can you describe what kind of conflicts you mean?

RM
Rana Muhammad Usman Nasir #109

Sorry got caught up in something, I manage to set the env variable, thanks alot.

Ni
Niko #110

Is it possible to use hetzner storage boxes for backups?

RM
Rana Muhammad Usman Nasir #111

We had voting in our Facebook groups, hetzner storage box didnt got much votes, but in later releases we can include it.

Go
Goran #112

We can use hetzner storage boxes as a SFTP destination, but it gives us errors.

Ni
Niko #113

What will work now? s3? what if I mount Storage Box via Samba/CIFS?

RM
Rana Muhammad Usman Nasir #114

I will create a video tutorial around it, basically as of this current release you can configure two sort of rclone backends from gui. However virtually you can configure any rclone backend manually from cli and CyberPanel will detect it and it will work just like a backup configured from UI.

RM
Rana Muhammad Usman Nasir #115

I was doing my final tests and ran into this

root@v2tests:~# sudo -u vtest8202 rustic -r rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website snapshots
using no config file (./rustic.toml doesn't exist)
enter repository password: 
[INFO] repository rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website: password is correct.
[INFO] using cache at /home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba

snapshots for (host [v2tests], label [], paths [/home/backup/v2testbackup.oso5ecswoq.cyberpanel.website/config.json,/home/v2testbackup.oso5ecswoq.cyberpanel.website,vB8ajaWbNO67rx.sql])
| ID       | Time                | Host    | Label | Tags | Paths                         | Files | Dirs |     Size |
|----------|---------------------|---------|-------|------|-------------------------------|-------|------|----------|
| 4054489e | 2023-04-11 21:48:15 | v2tests |       |      | /home/backup/v2testbackup.oso |  3275 |  400 | 88.7 MiB |
|          |                     |         |       |      | 5ecswoq.cyberpanel.website/co |       |      |          |
|          |                     |         |       |      | nfig.json                     |       |      |          |
|          |                     |         |       |      | /home/v2testbackup.oso5ecswoq |       |      |          |
|          |                     |         |       |      | .cyberpanel.website           |       |      |          |
|          |                     |         |       |      | vB8ajaWbNO67rx.sql            |       |      |          |
| 25d98af0 | 2023-04-11 22:01:42 | v2tests |       |      | /home/backup/v2testbackup.oso |  3285 |  412 | 89.1 MiB |
|          |                     |         |       |      | 5ecswoq.cyberpanel.website/co |       |      |          |
|          |                     |         |       |      | nfig.json                     |       |      |          |
|          |                     |         |       |      | /home/v2testbackup.oso5ecswoq |       |      |          |
|          |                     |         |       |      | .cyberpanel.website           |       |      |          |
|          |                     |         |       |      | vB8ajaWbNO67rx.sql            |       |      |          |
2 snapshot(s)

Then while i try to dump the database

root@v2tests:~# sudo -u vtest8202 rustic -r rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website dump 4054489e:vB8ajaWbNO67rx.sql --password ""
using no config file (./rustic.toml doesn't exist)
[INFO] repository rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website: password is correct.
[INFO] using cache at /home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba
[00:00:00]                                ████████████████████████████████████████          0/0                      [INFO] getting snapshot...
[00:00:00] reading index...               ████████████████████████████████████████          8/8                      Error: 

"vB8ajaWbNO67rx.sql" not found

it says vB8ajaWbNO67rx.sql not found, although its present in the snapshot.

aa
aawsome #116

Can you run rustic ls 4054489 to check the contents of this snapshot?
If in the output vB8ajaWbNO67rx.sql is missing: Can you tell me which command(s) you used to create that snapshot?

RM
Rana Muhammad Usman Nasir #117

@aawsome

command used for backup

mysqldump --defaults-extra-file=/home/cyberpanel/.my.cnf -u root --host=localhost --port 3306 --add-drop-table --allow-keywords --complete-insert --quote-names --skip-comments vB8ajaWbNO67rx | sudo -u vtest8202 rustic -r rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website backup --stdin-filename vB8ajaWbNO67rx.sql - --password "" --json 2>/dev/null

See the output below, looks like something really strange is going on in terms of path inside rustic, please note that this is a merged snapshot, so it contains other files as well. I see a stdin file at the end.

"home" 
"home/backup" 
"home/backup/v2testbackup.oso5ecswoq.cyberpanel.website" 
"home/backup/v2testbackup.oso5ecswoq.cyberpanel.website/config.json" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba/data" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba/data/30" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba/data/30/30e71493aa4fec41ef15e035c796cd997bf06114723fac8d2eef82626af9f6d0" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba/data/c8" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba/data/c8/c8d25fcb7ad99115f9a19db3f6b978757ebdc810ca9d4574f387ded15c4b5882" 
"home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba/index" 

"home/v2testbackup.oso5ecswoq.cyberpanel.website/public_html/xmlrpc.php" 
"stdin" 

Update: on another non-ubuntu system, everything work as expected

"home/cyberpanel.net/public_html/wp-includes/wp-diff.php" 
"home/cyberpanel.net/public_html/wp-links-opml.php" 
"home/cyberpanel.net/public_html/wp-load.php" 
"home/cyberpanel.net/public_html/wp-login.php" 
"home/cyberpanel.net/public_html/wp-mail.php" 
"home/cyberpanel.net/public_html/wp-settings.php" 
"home/cyberpanel.net/public_html/wp-trackback.php" 
"home/cyberpanel.net/public_html/xmlrpc.php" 
"home/cyberpanel.net/usman.jVceqzQ175.cyberpanel.website" 
"home/cyberpanel.net/usman.jVceqzQ175.cyberpanel.website/index.html" 
"qf4fza8IocVVqU.sql" 
"yKtbkp1THRlmBk.sql"

Update 2: Rather then picking the name from stdin, it is just calling it stdin

root@v2tests:~# sudo -u vtest8202 rustic -r rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website ls 03725241
using no config file (./rustic.toml doesn't exist)
enter repository password: 
[INFO] repository rclone:v2test:v2testbackup.oso5ecswoq.cyberpanel.website: password is correct.
[INFO] using cache at /home/v2testbackup.oso5ecswoq.cyberpanel.website/.cache/rustic/334bf000470161e8980cbeba79a40b2db566dd954ce0814df8b4740b9f871cba
[00:00:00]                                ████████████████████████████████████████          0/0                      [INFO] getting snapshot...
[00:00:00] reading index...               ████████████████████████████████████████         12/12                     

"stdin"
RM
Rana Muhammad Usman Nasir #119

Looks like does not happen on alma linux but on ubuntu only? Because on almalinux I see correct files.

JA
Joabe Arruda #120

Working together for open source. Very good! Looking forward to this major update. :slight_smile:

RM
Rana Muhammad Usman Nasir #123

Do you have any idea that why this exclude command is not actually working?

Logs folder still being backed up.

aa
aawsome #124

@usmannasir if you want to exclude all rusticbackup and logs folders, you should use the --glob option: --glob !rusticbackup --glob !logs. You may need some escaping or wrapping the pattern in quotes depending on how the command execution work.

The --exclude-if-present option ignores a path if it contains a file/dir with that specific name. It can be used e.g. to exclude cache dirs (using --exclude-if-present CACHEDIR.TAG) or to allow users to mark dirs not-to-backup by using --exclude-if-present .nobackup in the backup command and let users create themselves a .nobackup file in those dirs.

RM
Rana Muhammad Usman Nasir #125

logs and rusctic backup are present in {source} and I only wanted to exclude those specifi folders at {source}, do I need to give absolute path ?

aa
aawsome #126

Yes. If you don’t specify it as absolute path, every path with that name will be excluded.

aa
aawsome #128

You should use something like --glob "!{source}/logs". With --exclude-if-present your whole {source} dir will be excluded as within a directory named logs exists.

RM
Rana Muhammad Usman Nasir #130
mysqldump --defaults-extra-file=/home/cyberpanel/.my.cnf -u root --host=localhost --port 3306 --add-drop-table --allow-keywords --complete-insert --quote-names --skip-comments VTFs610I9oecK1 | sudo -u tests7889 rustic -r rclone:SHB:test.solveddoc.com backup --stdin-filename VTFs610I9oecK1.sql - --password "" --json 2>/dev/null
WARNING: Forcing protocol to  TCP  due to option specification. Please explicitly state intended protocol.

any idea why this warning is getting generated?

ZB
Zane Bernard #132

This is what ChatGPT thinks >>>

The warning message is generated because the command is using the “localhost” value for the --host option, but not specifying the protocol for the MySQL connection.

By default, the MySQL client will use the Unix socket protocol to connect to the MySQL server if the host is “localhost”. However, when specifying the “–port” option, the MySQL client will default to using the TCP protocol, which requires a network connection.

To avoid this warning message, you should explicitly specify the protocol to use for the MySQL connection by adding the “–protocol=TCP” option to the command.

For example:

mysqldump --defaults-extra-file=/home/cyberpanel/.my.cnf -u root --host=localhost --protocol=TCP --port 3306 --add-drop-table --allow-keywords --complete-insert --quote-names --skip-comments VTFs610I9oecK1 | sudo -u tests7889 rustic -r rclone:SHB:test.solveddoc.com backup --stdin-filename VTFs610I9oecK1.sql - --password "" --json 2>/dev/null

This should ensure that the intended protocol is used for the MySQL connection, and prevent the warning message from being generated.

RM
Rana Muhammad Usman Nasir #133

Yes just fixed this issue.

CX
Christos Xanthou #135

Im using contabo object storage(S3)
I saw that still is not supported:

Will first initiate backup repo..
Fatal: create repository at s3:s3.amazonaws.com/netgen.gr failed: client.BucketExists: The AWS Access Key Id you provided does not exist in our records.


Repo s3:s3.amazonaws.com/2353f42c14eabd2b0904db23c46cd3c7 initiated for netgen.gr.
Meta data is ready..
Backing up databases..
Fatal: unable to open config file: Stat: The AWS Access Key Id you provided does not exist in our records.
Is there a repository at the following location?
s3:s3.amazonaws.com/netgen.gr
. [5009].

Do we have a time shecdude when will be supported?

Lu
Lucas #136

Hello,
I’ve been using cyberpanel but I’m having problems with the scheduled incremental backup.

CentOS 7 64-bit with CyberPanel (hostinger)

Whenever it runs, my server uses a LOT of CPU and stays that way forever until I reboot.

I would like to understand the reason for this problem and try to fix it.

Some prints below for better understanding:

image

image

Thanks!

JA
Joabe Arruda #141
ik
ikk157 #142

@usmannasir We need a one click full backup option.

And when I say Full I mean everything, including: The cyberpanel users, ACLs, Packages, all of the websites and their configs, docker and everything to do with it, literally everything.

So that we can backup the entire cyberpanel server data into one singular file. This is especially useful when upgrading the OS. As currently, we have to make a seperate backup per website, then restore each one seperately after re-installing the OS and cyberpanel, then manually having to re-create each user, each ACL, each package, and manually assign the websites to the correct users and ACLs, manually re-create all of the docker containers. It is an absolute nightmare!

Being able to back all of this up into one file with a few clicks, and being able to restore everything from that single file would solve this. Many panels already do this, and it’s the one feature I genuinely always wished cyberpanel had. I got super excited when I saw backupsv2 pop up after updating cyberpanel, as I thought this finally brought this functionality, only to be disappointed.

CX
Christos Xanthou #143

They are asking for 139 dollars for all the plugins, but in my opinion, and I believe many others share this view, I would willingly pay DOUBLE if I had something that actually worked and was not full of bugs or incompatibilities. For instance:

  1. For the schedule remotely backup, I want to connect it to the Contabo object storage, which has been incompatible for quite some time now.
  2. WordPress Manager: Coming from a similar plugin in Plesk, I assure you it is NO comparison. Essential features are missing, they could simply copy the innovations from the very successful Plesk plugin. This one has no comparison and I can detail those things that it doesn’t have or those that are not working.
  3. SSL: I have my CyberPanel as a subdomain and for some time now, the SSL has been broken and doesn’t seem to get fixed, regardless of any changes I make.
  4. Unfriendly CyberPanel update system: It could be done with a simple button, instead of saying that the git code differs from the stable version… And many others.

So, I can assure you that if you fix these two crucial things, I would pay double for the one-time purchase…

NW
Nathan Waelkens #144

I agree. I bought the plugins too, but wished it just works.

My feedback would be: test test test! Test every feature thorough. Nothing really works out of the box. I prefer less features that just work or indeed - double the price - as long as it works. I’m sure the community can help in beta testing features. Every promise that isn’t met is a dissapointment. People can have a few, but will end up walking away. Backups v2: doesn’t work for me. I need to figure out now what this means: [06.02.2023_10-21-03:FAILED] Failed during config generation, Error: Expecting value: line 1 column 1 (char 0)[404]. That’s after the issue I had with not being able to setup FTP accounts.

Still enthusiastic and really see lots of potential. But please, give the user experience and the usertests more attention. I have now 3 backup features in my CP. None really usable so far.

Perhaps the community can help. Ask me for a feature to test: I’ll record it while using it and you can see what I’m running into. You have users and a community.

I hope to have kept it a bit constructive. :slight_smile: All the best. Life is a learning curve.

mr
mrwebng #145

There are so many things wrong with CyberPanel right now. A user who is not an admin should not see certain things. As with all the addons that the admin does not subscribe to, this is very awkward and will force me to have a change of mind before some high-profile customers start moving their websites to my server. Something should be ddone ASAP or provide the ability to hide them.

MA
Md. Abdul Hakim #146

I know how to hide them xD… Just do the Follows

Open
/usr/local/CyberCP/baseTemplate/templates/baseTemplate

There u got the Sidebar. Now if u know basic html u should be able to figure out which is the part of that specific menu item, then after u have figure the start and the end of the specific menu item u just need to wrap then with
{% if admin %}
in the start of the menu item and in the end
{% endif %} thats it Xd

LM
Luciano Milani #147

@usmannasir @josephgodwinke and all staff,

Good morning cyberpanel team, I think this solution can be much more effective and better than the current solution (from what I could tell from the video), but it is a paid solution that requires the purchase of the addon.

The free CyberPanel solution is very limited and unintuitive, I think it is the biggest flaw and the most sensitive of the CP functions.

There is no certainty about how it works, no consistency about retention and it is not a simple and intuitive configuration process (all users/customers that were used to CentosWEBPanel, CPanel and CloudPanel, have a hard time understanding the flow)

Then there is still something that is terrible and makes you think about changing… (it may seem hard to hear this but when I made the choice to use CyberPanel it was through much analysis and seeing several people and videos talking good about the system … however the BKPs are fundamental in the ecosystem and if there is no appreciation and respect for those who do not have the paid plan I find it difficult to maintain the choice … When you started the project there were not so many paid tools… now I honestly believe that the development team only cares about the paid addons and their development… leaving the basic functionalities without commercial value to oblivion.

Reflect on this strategy, especially when you started this journey with a panel focused on being free and powerful.

think about it!

LM
Luciano Milani #149

Its possible create conection and remote bkps in MINIO???

CX
Christos Xanthou #150

Good evening again,
I thought to wait a bit in case the communication between contabo S3 and obs would be integrated, but nothing so far.
I chose to buy a new SFTP server from Hetzner to ensure compatibility for remote backups, and now I see that I can’t even connect the server here, which is unbelievable guys.
The backup is the most important aspect of this service, can you tell me what’s wrong? I am supposed to have the premium version of cyberpanel, if this is how it is I’m going back to Plesk for peace of mind.

image

Sign in to reply