Nice one. I just did that and removed the /home/persistent-domain-name folder again as well - will have to wait to see if it persists!
@usmannasir There’s also many other domains that I deleted a while ago in /etc/letsencrypt/live. Should these not be deleted when you delete a website?