Technical Issue Report: Arabic Characters Corruption in CyberPanel File Manager
Issue Description
When opening files containing Arabic text via the CyberPanel File Manager, some characters are missing or corrupted, while the same file displays correctly when accessed via SSH.
Symptoms
-
Correct Output (via SSH):
cat /home/example.com/public_html/text.txt
Output:
"كيف الحال يا من للحب مجال والسلام ختام"
-
Corrupted Output (via File Manager):
{status: 1, fileContents: "كيف الحال يا من للحب مجال والسلام خام", cont1: 37}
Missing character:
"ت"
Debugging Findings
- The
outputExecutioner()
function retrieves file contents but loses some Arabic characters when processing the output. len()
returns an incorrect count, confirming data loss.- SSH works fine, meaning the issue is not in file storage but in how CyberPanel processes the output.
Attempted Fixes (Unsuccessful)
-
Added UTF-8 encoding explicitly:
p = subprocess.Popen(..., encoding='utf-8', errors='strict', env=env_vars)
→ No improvement.
-
Forced UTF-8 locale:
env_vars['LANG'] = 'en_US.UTF-8' env_vars['LC_ALL'] = 'en_US.UTF-8'
→ Still losing characters.
Request for Help
Since manual fixes haven’t resolved the issue, I’m seeking assistance from the CyberPanel community to:
Possible Investigation Paths
-
Is there an encoding mismatch between:
- The system locale (
locale
command) - Python’s default encoding (
sys.getdefaultencoding()
) - The subprocess environment?
- The system locale (
-
Does the issue occur before or after
outputExecutioner()
?- If I log the raw bytes before decoding, are they correct?
-
Is there a middleware (like gunicorn, nginx, or uWSGI) modifying the response?
-
Does this happen with other Unicode characters (e.g., emojis, Chinese)?
Questions for the Community
- Has anyone faced similar encoding issues in CyberPanel?
- Are there known fixes for Arabic/Persian/Urdu text handling?
- Should we modify the subprocess call or change how the response is sent to the frontend?
How to Reproduce
- Create a file with Arabic text.
- Open it via File Manager → Characters are missing.
- Check via SSH → Text is correct.
Conclusion
The issue appears to be in how CyberPanel processes command output, not in the file itself. Since my attempts to enforce UTF-8 failed, I’m looking for insights from developers who may have encountered this before.
Any suggestions or debugging tips would be greatly appreciated!
Relevant Code
The current outputExecutioner()
implementation:
def outputExecutioner(command, user=None, shell=None, dir=None, retRequired=None):
env_vars = os.environ.copy()
env_vars['LANG'] = 'en_US.UTF-8'
env_vars['LC_ALL'] = 'en_US.UTF-8'
try:
# ... (subprocess call with UTF-8 settings)
output = p.communicate()[0].decode("utf-8") # Still loses Arabic chars
return output
except Exception as e:
logging.error(f"Encoding error: {e}")
return ""
Operating System Information
CyberPanel 2.4.2 - The problem existed even before the update.
AlmaLinux 8
Looking forward to your ideas!