Now I got it working. First time configuring Litespeed so didn’t find it first, but it is quite obvious when I found it.
So what I did was looked from litespeed config under “Listeners” and I noticed Listeners are only set for IPv4. So I just added IPv6 listeners and used same configuration as in IPv4.
Now it works with IPv4 and IPv6. Here is some screenshots to help others:
and remember to add Virtual Host Mappings:
