Those who Can't Access SSH and Still can't recover their Server!

Hello
Hope everyone doing well.

I have seen so many users still couldn’t recover their servers.

  1. NO Access to SSH ( Connection Refused or Timeout )
  2. Unable to Access cyberpanel
  3. Unable to Access Litespeed
  4. Not able to Login to Server Console or Recovery Mode

After analysing 100+ servers in such situations, I have found that the hacker modified the netplan, network files and that’s why the server OS couldn’t connect to the Internet itself and nothing works!

There we can do these things to Initialize and recover your server ( 90% chances we can )

1. Login Via Recovery Mode or SSH

Hetzner Guide: How to Login using CHROOT ( Only via Recovery Mode )
Hetzner Guide: How to use the VNC in Hetzner
Contabo Guide: How to Login using VNC

It should run as the root, if not, getting any issues like BIN/BASH not found then kindly ask chatgpt to fix this issue and try again to login as root.

STEP 1: Check Processes

Use these commands

htop

if you found some unknown processes then it means your server is corrupted. Proceed to next step.


STEP 2: Run the Kinsing Removal Script

IMPORTANT

Run this script to remove kingsing or similar some processes which are using your cpu at max. It only can be possible if Internet is working in your server.

wget -qO- https://code.luveedu.com/kinsing-cleanup.sh | bash

Credits: ManagingWP

The reason behind changing the URL is because the Hacker Blocked access to .github.com | So don’t mind buddy.


STEP 3: Check Resolvers

IMPORTANT

Now check the resolvers to a perfect one. Like Google, Cloudflare or Adguard etc.

nano /etc/resolv.conf

For Google use 8.8.8.8 / 8.8.4.4


STEP 4: Generate NetPlans Again

CAUTION: For me I was using Hetzner and Hivelocity so I want able to reset it. You can take proper assistance about it from your VPS Provider and also double check these data with the provider.

First Check the IP is working or not

ping 8.8.8.8

The result should not contain text like “Unable to reach | Operation not permitted”


Then check the IP config and IP Route

ip a

ip route

The IPV4 should be present in the ip a command output and the Gateway like eg. 172.1.1.1/24 something like should be present in the ip route command output.

If there’s nothing matching my words then it means the server netplans are modified.
So run these commands below to generate / reset the netplans.

netplan generate

netplan apply

ip a

ip route

ping 8.8.8.8

If still says that “Operation not permitted | Unreachable server | Unable to resolve to host”
Then Proceed to next step.


STEP 5: Final Step ( Risk )

CAUTION: This step is very risky, I have tested it in 50+ servers that’s why I have written this. You can try at your own

Stop Cyberpanel & Litespeed Web Server

systemctl stop lscpd

systemctl stop lsws

Now we can kill all the processes. But this will not kill the current ssh or recovery shell process.

kill -9 -1

Then run the Kinsing Removal Script again. If able to run. ( Hopefully we can )

Now if You can Install or run apt then install ClamAV

yum or apt install clamav -y

You can run the scan later also, because it will take much time to complete. Or if you have time you can run it.

clamscan -r /

Then Reboot Your system. Once done Instantly Login using SSH and run these commands.

systemctl start lscpd

systemctl start lsws

Then Login to Cyberpanel then do all of these below.

  1. Update Cyberpanel: 02 - Upgrading CyberPanel
  2. Block Access to SSH, IMAP, SMTP, CYBERPANEL, LSWS Ports using firewall, whitelist your IP before. ( Caution! You might get locked! )
  3. Run the ClamAV ( If not run before )
  4. Run the Kinsing Removal Script Once again
  5. Check the Resolver and Update the Server Files apt update & apt upgrade.
  6. Cyberpanel Team fixed the backup Issues and now backups also working. So instantly take backups and migrate to somewhere else.
  7. Then Buy a Good Protection Like Imunify, BitNinja or cpGuard. I was also using cpGuard in my some servers and it kinda worked on 2 3 servers because it blocked the processes using too much resource at the very beginning.
  8. Run the Kinsing Removal Script many times and it actually helps. Thanks ManagingWP for this.
  9. Also Change the SSH Port.
  10. In secure SSH > Keys Make sure only your added Key is showing.
  11. Change Root password, Cyberpanel Password, Admin Password, Any password you have asap.
  12. Enable Cloudflare in all domains.
  13. More steps are mentioned in the references.

STEP 6: Check for ROOTKITS

IMPORTANT

You should check your server for rootkit injections because they can silently hide in your server, and after 1 week or 1 month can easily attack your site again and can lock or encrypt your file again.

Follow this guide to Install Rkhunter and Check for Rootkits.


STEP 7: Prevent Threats

IMPORTANT

As this is a open source platform and a technical product there can be high alerts. So prevent future attacks and this kind of issues.

Follow these things.

  1. Enable Cloudflare in all the domains
  2. Disable Cyberpanel Portal when you are not using - systemctl stop lscpd - Your DNS, Emails everything will work normally. And for files use FTP with Port Open by any Static IP or for a timeframe.
  3. Go to Litespeed Web console > Server configuration > Security > Access Control > Allow only Cloudflare IP list in a comma separated list & Your static IP. If you don’t have static IP then don’t do this.


2. No Recovery Mode or Access

They can message below their situation.



3. Not Access to Anything, Totally Locked

The best resolution is to Give your server a 1 hour of total rest. Then contact your provider to stop internet connectivity for 1 hour and also close all ports to the server for 1 hour. Because if you can’t access using chroot or ssh or if the steps I mentioned earlier then it means your files are going to locked or encrypted, so stop the Agent in the server to do so.



4. Files are Encrypted or Locked

I can see Cyberpanel Suggested some guides and urls also in my reference you can follow them. Else backup and restore the best solution.



And I can understand the situation of everyone in Cyberpanel Team they are working without sleeping. I can understand this and if they allow me then I can help users recovering if users can’t have information about Linux and Cyberpanel. Or they can also do the steps I mentioned because I did these steps.

  1. Need more informative Guide?
  2. Is this Guide Wrong or Manipulating?
  3. Should I capture a video guide?

But always remember: Linux Commands are subject to market risk, read all guides and scam related documents carefully.

Thank You :slightly_smiling_face:



If someone still getting issues? Couldn’t recover, we have a bunch of server experts to help you resolve, recover your problems For Free If you like. You can contact us.


References

bug fix: pre-auth · usmannasir/cyberpanel@5b08cd6 · GitHub

cyberpanel 0day leaked attack script · GitHub

Manually applying the Patch via Rescue Mode

address some security concerns · usmannasir/cyberpanel@ba0831f · GitHub

https://docs.hetzner.com/robot/dedicated-server/troubleshooting/hetzner-rescue-system/

https://cyberpanel.net/blog/letter-about-cyberpanel-breech-2024

Critical Security Alert: Vulnerable CyberPanel Instance Detected on Your Network

Solution for protecting yourself from future threats

From Cyberpanel team: We’re here to help—feel free to reach out anytime by emailing us at: [email protected]

7 Likes

Thank you very much! :heart:

1 Like

This is helpful. My server were also locked due to interference with network settings, I managed to get my data out using recovery mode though running mysql in safe mode using chroot on mounted image from GRML

https://sunng.info/blog/db-dump-with-chroot.html

I suggest wiping the system clean and reinstalling OS/Cyberpanel and then restoring your sites. Something people don’t know you can change your cyberpanel port as well which would stop all these script kiddies from even trying to attack cyberpanel. Also there is no need for cyberpanel to keep running, just shut it down when you don’t need it - your sites will run fine.

1 Like

I am waiting for Cyberpanel team to fix the backup issue and file manager issues. So once they fixed all of these then we can backup and restore easily, if got access.

thank you very much Ariyan

your article also helped me a lot to fix the problem

Best Regards

1 Like

Thanks Ariyan for the detailed explanation. I am still stuck and tried your process but still nothing. I need your help to check my server. Let me know if I can send you a DM. Thanks

1 Like