You can't force people to turn up to the site, but what you can do is make sure that the site is welcoming to new visitors of all kinds.

For example, make sure that things like sexist names and avatars (pictures of scantily clad women for example) are known to be inappropriate and will be removed when found. Such things are already against the Stack Exchange terms of service, but it would be worth reminding people - particularly if you have recently had some issues in this area (NOTE: I'm not saying you have, just that you should take preventative action).

I know that sounds like I'm stating the obvious but if anyone comes to the site and their first impression is that it's a closed club and they're not welcome then that's certain to ensure that they don't come back.