v1.9 Build 3E - python2 binary removed, but crontab not updated & Bandwidth Script not working

[63DYxLP1]

Provider: KVM VPS, Self Hosted
Virtualization: KVM
System: CentOS 7 (All Updated)
OLS/Enterprise: OLS
Official Mirror: Yes
Memcache: No
Redis: Yes

Hi Guys,

I recently upgraded my CyberPanel installation to v1.9 Build 3E and after a few days I realized my backups were no longer operational.

In checking the crontab I noticed the backup entries were pointing to “python2” binary which appears to have been removed as part of the upgrade and replaced with “python3”.

0 * * * * root python /usr/local/CyberCP/plogical/findBWUsage.py
0 * * * * root /usr/local/CyberCP/postfixSenderPolicy/client.py hourlyCleanup
0 0 1 * * root /usr/local/CyberCP/postfixSenderPolicy/client.py monthlyCleanup
0 2 * * * root /usr/local/CyberCP/plogical/upgradeCritical.py
0 2 * * * root /usr/local/CyberCP/bin/python2 /usr/local/CyberCP/plogical/renew.py
0 12 * * * root /usr/local/CyberCP/bin/python2 /usr/local/CyberCP/IncBackups/IncScheduler.py Daily
0 0 * * 0 root /usr/local/CyberCP/bin/python2 /usr/local/CyberCP/IncBackups/IncScheduler.py Daily

I have confirmed my backups are back to running, so that issue has been resolved, however, when running “findBWUsage.py” I am getting the following error now:

[root@web01 ~]# python /usr/local/CyberCP/plogical/findBWUsage.py
Traceback (most recent call last):
File “/usr/local/CyberCP/plogical/findBWUsage.py”, line 2, in
from plogical import CyberCPLogFileWriter as logging
ImportError: No module named plogical

The first part of my post is just an FYI, second part I would like any advice y’all can share to get me back to an operational state.

Thank you!

[Zalcig]

I had some similiar case with backups, then I just did a fresh install then took the problematic .py file in my case it was backuplocal.py or sth like that and replaced where it was problem solved. You can do the same with replacing the problematic .py file with a fresh install

[63DYxLP1]

Type your comment> @neweracloud said:

Don’t want to sound condescending but I would not have upgraded without taking a snapshot of the server first. Then if above occurred I could re-instate the snapshot and use updraft plugin to save wp sites for re-install after upgrade.
That said, if all is working and your only site backups are CyberPanel there must be a way to get them back.
ie/ re-install CyberPanel if the backups are stable and take it from there.

I have a full backup (many in this case) of the entire VM from before the upgrade was performed as well as the nightly backups created by CyberPanel itself.

As noted, my backups are fully operational once I had updated the crontab entries to use the new path for CyberPanel’s packaged python version (ie. python2 → python).

My post here is to serve as a bug report (Hence why I posted in the bug report section) as well as to ask for help with the bandwidth script.

[63DYxLP1]

@Zalcig said:
I had some similiar case with backups, then I just did a fresh install then took the problematic .py file in my case it was backuplocal.py or sth like that and replaced where it was problem solved. You can do the same with replacing the problematic .py file with a fresh install

The findBWUsage.py is the latest version from the installation repo.

[63DYxLP1]

Type your comment> @neweracloud said:

Request for ideas. comments, problem seemed solved, bug report. It’s all good @ #Newera2020

What?

[whattheserver]

Thanks for this i was able to identify why my backups were not working.

Also @63DYxLP1 the issue with the findbw is now fixed in 1.9.4 per @usmannasir commit:

update path

The crons with proper paths and notifications set to dev/null so root mail account or forwarder is not spammed.
0 * * * * root /usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/findBWUsage.py >/dev/null 2>&1
0 * * * * root /usr/local/CyberCP/bin/python /usr/local/CyberCP/postfixSenderPolicy/client.py hourlyCleanup >/dev/null 2>&1
0 0 1 * * root /usr/local/CyberCP/bin/python /usr/local/CyberCP/postfixSenderPolicy/client.py monthlyCleanup >/dev/null 2>&1
0 2 * * * root /usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/upgradeCritical.py >/dev/null 2>&1
0 2 * * * root /usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/renew.py >/dev/null 2>&1
0 12 * * * root /usr/local/CyberCP/bin/python /usr/local/CyberCP/IncBackups/IncScheduler.py Daily >/dev/null 2>&1
0 0 * * 0 root /usr/local/CyberCP/bin/python /usr/local/CyberCP/IncBackups/IncScheduler.py Weekly>/dev/null 2>&1

Ending up writing a script to add the crons if they did not already exist so I could rapidly add them to the VPS which were missing.

#!/bin/bash

nano add_missing_cyberpanel_crons.sh

#Install findBWUsage cron if missing
command=“root /usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/findBWUsage.py”
job=“0 * * * * $command >/dev/null 2>&1”
cat <(grep -Fiv “$command” <(crontab -l)) <(echo “$job”) | crontab -

#Install postfix hourly cron if missing
command=“root /usr/local/CyberCP/bin/python /usr/local/CyberCP/postfixSenderPolicy/client.py hourlyCleanup”
job=“0 * * * * $command >/dev/null 2>&1”
cat <(grep -Fiv “$command” <(crontab -l)) <(echo “$job”) | crontab -

#Install postfix monthyl cron if missing
command=“root /usr/local/CyberCP/bin/python /usr/local/CyberCP/postfixSenderPolicy/client.py monthlyCleanup”
job=“0 0 1 * * $command >/dev/null 2>&1”
cat <(grep -Fiv “$command” <(crontab -l)) <(echo “$job”) | crontab -

#Install upgradeCritical cron if missing
command=“root /usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/upgradeCritical.py”
job=“0 2 * * * $command >/dev/null 2>&1”
cat <(grep -Fiv “$command” <(crontab -l)) <(echo “$job”) | crontab -

#Install upgradeCritical cron if missing
command=“root /usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/renew.py”
job=“0 2 * * * $command >/dev/null 2>&1”
cat <(grep -Fiv “$command” <(crontab -l)) <(echo “$job”) | crontab -

[63DYxLP1]

@whattheserver - Awesome, thank you!

[whattheserver]

My pleasure. I noticed that there was a slight issue as the user does not need to be specified when using the proper crontab.

As you already ran this here are two commands to fix this and remove the invalid crons in /etc/crontab.

sed -i ‘/CyberCP/d’ /etc/crontab
sed -i ‘s/root //g’ /var/spool/cron/root

I submitted a commit to fix this in both the install and the minor fix to upgrade.py usman pushed yesterday so this will be automatically fixed and checked for in all upgrades.

Remove invalid system crons setup via /etc/crontab. Adjust install.py to install crons to the root user crontab. Fix upgrade.py to ensure crons setup properly under root crontab. Remove duplicate old crons for Incscheduler section.

Before:
root@server [/root]# cat /etc/crontab
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root

For details see man 4 crontabs

Example of job definition:

.---------------- minute (0 - 59)

| .------------- hour (0 - 23)

| | .---------- day of month (1 - 31)

| | | .------- month (1 - 12) OR jan,feb,mar,apr …

| | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat

| | | | |

* * * * * user-name command to be executed

0 * * * * root python /usr/local/CyberCP/plogical/findBWUsage.py
0 * * * * root /usr/local/CyberCP/postfixSenderPolicy/client.py hourlyCleanup
0 0 1 * * root /usr/local/CyberCP/postfixSenderPolicy/client.py monthlyCleanup
0 2 * * * root /usr/local/CyberCP/plogical/upgradeCritical.py
0 12 * * * root /usr/local/CyberCP/bin/python2 /usr/local/CyberCP/IncBackups/IncScheduler.py Daily
0 0 * * 0 root /usr/local/CyberCP/bin/python2 /usr/local/CyberCP/IncBackups/IncScheduler.py Daily
0 2 * * * root /usr/local/CyberPanel/bin/python /usr/local/CyberCP/plogical/renew.py
root@server [/root]#

Showing a preview without modifiying file. Notice it remove just the bad crons in preview mode
root@server [/root]# sed ‘/CyberCP/d’ /etc/crontab
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root

For details see man 4 crontabs

Example of job definition:

.---------------- minute (0 - 59)

| .------------- hour (0 - 23)

| | .---------- day of month (1 - 31)

| | | .------- month (1 - 12) OR jan,feb,mar,apr …

| | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat

| | | | |

* * * * * user-name command to be executed

Stuff was not modified in original file
root@server [/root]# cat /etc/crontab
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root

For details see man 4 crontabs

Example of job definition:

.---------------- minute (0 - 59)

| .------------- hour (0 - 23)

| | .---------- day of month (1 - 31)

| | | .------- month (1 - 12) OR jan,feb,mar,apr …

| | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat

| | | | |

* * * * * user-name command to be executed

0 * * * * root python /usr/local/CyberCP/plogical/findBWUsage.py
0 * * * * root /usr/local/CyberCP/postfixSenderPolicy/client.py hourlyCleanup
0 0 1 * * root /usr/local/CyberCP/postfixSenderPolicy/client.py monthlyCleanup
0 2 * * * root /usr/local/CyberCP/plogical/upgradeCritical.py
0 12 * * * root /usr/local/CyberCP/bin/python2 /usr/local/CyberCP/IncBackups/IncScheduler.py Daily
0 0 * * 0 root /usr/local/CyberCP/bin/python2 /usr/local/CyberCP/IncBackups/IncScheduler.py Daily
0 2 * * * root /usr/local/CyberPanel/bin/python /usr/local/CyberCP/plogical/renew.py
root@server [/root]#

Now were going to safely only remove the bad cyberpanel crons from this file and root user in the crontab for the root user as its not needed.
root@server [/root]# sed -i ‘/CyberCP/d’ /etc/crontab
root@server [/root]# cat /etc/crontab
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root

For details see man 4 crontabs

Example of job definition:

.---------------- minute (0 - 59)

| .------------- hour (0 - 23)

| | .---------- day of month (1 - 31)

| | | .------- month (1 - 12) OR jan,feb,mar,apr …

| | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat

| | | | |

* * * * * user-name command to be executed

root@server [/root]#

[dyrer]

Hello, cron stats are fixed?
I did a new clean install and cron are not working
I tried update and got a lot of errors
I tested with Centos 7,8 and Ubuntu 18.04