First thing to keep in mind is that purpose of hot questions feature is established pretty firmly and that it's not really about helping in your community growth. The feature is there only to show network wide audience entertaining / interesting questions. If you're interested, refer MSE for more details on that: What is the Goal of “Hot Network Questions”?What is the Goal of “Hot Network Questions”?
Because of this, you should not generally expect this feature to work in favor of your community. You can refer examples at ProgrammersProgrammers and Workplace to find more details about that. "Tons and tons of people visited the site... but very few decided to stick around..." "flash in the pan..." - that's what you can expect (unless you put dedicated effort into adapting it to your needs, but more on that below).
So, if you want to drop the off-topic question from the hot list, 1) try to get it closed sooner, 2) vote down and 3) abstain of answering. For the latter, additional feature to consider is protectionprotection from new users' answers but more on that later.
High exposure of the question to the inexperienced visitors tends to bring answer quality issuesanswer quality issues:
Stack Exchange team established a dedicated automatic protection featureautomatic protection feature intended to help with issues like this. If voting on your site is relatively anemic and if site doesn't get many visitors "armed" with association bonus reputationassociation bonus reputation (like it is for example at Stack Overflow), expect basic protection offered by this feature suffice to save you from most troublesome effects of popularity. If voting at your site differs from that one you may find it a bit too fragilea bit too fragile - I am not aware of general solution for that, you'd probably have to discuss it at site meta if it grows into substantial problem.
Consider actively editing, checking for repetitive and low quality answers, up to the point of involving moderators in cleaning up and deletion of troublesome content. A solid guidance on the latter was given a while ago by Jeff Atwoodgiven a while ago by Jeff Atwood: