Block Direct IP Access

How to block direct IP access and return 403?

I think you can do it via CSF but im wondering is it possible without CSF.

CSF refused my smtp connections thats why i can’t use CSF …