Hello
Hope everyone doing well.
I have seen so many users still couldn’t recover their servers.
- NO Access to SSH ( Connection Refused or Timeout )
- Unable to Access cyberpanel
- Unable to Access Litespeed
- Not able to Login to Server Console or Recovery Mode
After analysing 100+ servers in such situations, I have found that the hacker modified the netplan, network files and that’s why the server OS couldn’t connect to the Internet itself and nothing works!
There we can do these things to Initialize and recover your server ( 90% chances we can )
1. Login Via Recovery Mode or SSH
Hetzner Guide: How to Login using CHROOT ( Only via Recovery Mode )
Hetzner Guide: How to use the VNC in Hetzner
Contabo Guide: How to Login using VNC
It should run as the root, if not, getting any issues like BIN/BASH not found then kindly ask chatgpt to fix this issue and try again to login as root.
STEP 1: Check Processes
Use these commands
htop
if you found some unknown processes then it means your server is corrupted. Proceed to next step.
STEP 2: Run the Kinsing Removal Script
IMPORTANT
Run this script to remove kingsing or similar some processes which are using your cpu at max. It only can be possible if Internet is working in your server.
wget -qO- https://code.luveedu.com/kinsing-cleanup.sh | bash
Credits: ManagingWP
The reason behind changing the URL is because the Hacker Blocked access to .github.com | So don’t mind buddy.
STEP 3: Check Resolvers
IMPORTANT
Now check the resolvers to a perfect one. Like Google, Cloudflare or Adguard etc.
nano /etc/resolv.conf
For Google use 8.8.8.8 / 8.8.4.4
STEP 4: Generate NetPlans Again
CAUTION: For me I was using Hetzner and Hivelocity so I want able to reset it. You can take proper assistance about it from your VPS Provider and also double check these data with the provider.
First Check the IP is working or not
ping 8.8.8.8
The result should not contain text like “Unable to reach | Operation not permitted”
Then check the IP config and IP Route
ip a
ip route
The IPV4 should be present in the ip a
command output and the Gateway like eg. 172.1.1.1/24 something like should be present in the ip route
command output.
If there’s nothing matching my words then it means the server netplans are modified.
So run these commands below to generate / reset the netplans.
netplan generate
netplan apply
ip a
ip route
ping 8.8.8.8
If still says that “Operation not permitted | Unreachable server | Unable to resolve to host”
Then Proceed to next step.
STEP 5: Final Step ( Risk )
CAUTION: This step is very risky, I have tested it in 50+ servers that’s why I have written this. You can try at your own
Stop Cyberpanel & Litespeed Web Server
systemctl stop lscpd
systemctl stop lsws
Now we can kill all the processes. But this will not kill the current ssh or recovery shell process.
kill -9 -1
Then run the Kinsing Removal Script again. If able to run. ( Hopefully we can )
Now if You can Install or run apt then install ClamAV
yum or apt install clamav -y
You can run the scan later also, because it will take much time to complete. Or if you have time you can run it.
clamscan -r /
Then Reboot Your system. Once done Instantly Login using SSH and run these commands.
systemctl start lscpd
systemctl start lsws
Then Login to Cyberpanel then do all of these below.
- Update Cyberpanel: 02 - Upgrading CyberPanel
- Block Access to SSH, IMAP, SMTP, CYBERPANEL, LSWS Ports using firewall, whitelist your IP before. ( Caution! You might get locked! )
- Run the ClamAV ( If not run before )
- Run the Kinsing Removal Script Once again
- Check the Resolver and Update the Server Files apt update & apt upgrade.
- Cyberpanel Team fixed the backup Issues and now backups also working. So instantly take backups and migrate to somewhere else.
- Then Buy a Good Protection Like Imunify, BitNinja or cpGuard. I was also using cpGuard in my some servers and it kinda worked on 2 3 servers because it blocked the processes using too much resource at the very beginning.
- Run the Kinsing Removal Script many times and it actually helps. Thanks ManagingWP for this.
- Also Change the SSH Port.
- In
secure SSH > Keys
Make sure only your added Key is showing. - Change Root password, Cyberpanel Password, Admin Password, Any password you have asap.
- Enable Cloudflare in all domains.
- More steps are mentioned in the references.
STEP 6: Check for ROOTKITS
IMPORTANT
You should check your server for rootkit injections because they can silently hide in your server, and after 1 week or 1 month can easily attack your site again and can lock or encrypt your file again.
Follow this guide to Install Rkhunter and Check for Rootkits.
STEP 7: Prevent Threats
IMPORTANT
As this is a open source platform and a technical product there can be high alerts. So prevent future attacks and this kind of issues.
Follow these things.
- Enable Cloudflare in all the domains
- Disable Cyberpanel Portal when you are not using -
systemctl stop lscpd
- Your DNS, Emails everything will work normally. And for files use FTP with Port Open by any Static IP or for a timeframe. - Go to Litespeed Web console > Server configuration > Security > Access Control > Allow only Cloudflare IP list in a comma separated list & Your static IP. If you don’t have static IP then don’t do this.
2. No Recovery Mode or Access
They can message below their situation.
3. Not Access to Anything, Totally Locked
The best resolution is to Give your server a 1 hour of total rest. Then contact your provider to stop internet connectivity for 1 hour and also close all ports to the server for 1 hour. Because if you can’t access using chroot or ssh or if the steps I mentioned earlier then it means your files are going to locked or encrypted, so stop the Agent in the server to do so.
4. Files are Encrypted or Locked
I can see Cyberpanel Suggested some guides and urls also in my reference you can follow them. Else backup and restore the best solution.
And I can understand the situation of everyone in Cyberpanel Team they are working without sleeping. I can understand this and if they allow me then I can help users recovering if users can’t have information about Linux and Cyberpanel. Or they can also do the steps I mentioned because I did these steps.
- Need more informative Guide?
- Is this Guide Wrong or Manipulating?
- Should I capture a video guide?
But always remember: Linux Commands are subject to market risk, read all guides and scam related documents carefully.
Thank You
If someone still getting issues? Couldn’t recover, we have a bunch of server experts to help you resolve, recover your problems For Free If you like. You can contact us.
References
address some security concerns · usmannasir/cyberpanel@ba0831f · GitHub
https://docs.hetzner.com/robot/dedicated-server/troubleshooting/hetzner-rescue-system/
https://cyberpanel.net/blog/letter-about-cyberpanel-breech-2024
Critical Security Alert: Vulnerable CyberPanel Instance Detected on Your Network
From Cyberpanel team: We’re here to help—feel free to reach out anytime by emailing us at: [email protected]