I have managed forums that follow both policies - keep every account ever made and delete inactive users. There are perceived benefits to both, but if the account is unused it probably doesn't matter one way or the other what you decide to do with the accounts.
Delete Unused accounts
The benefit of deleting these unused accounts is a cleaner user base. You have potential spammers and bad actors removed from your system. I didn't run the process to remove old users on a scheduled basis. Usually it'd be run after a new spam wave hit the forum.
I started doing this because many of the accounts I was getting hit with were sleeper accounts. They'd register and not post for a month or two. Same day registration and postings were not as common as you'd expect, in my case. Thus, deleting the accounts that were never used (never even logged in) within a month solved many of my recurring spam problems on that particular community.
The downside of this, is that you occasionally remove a user that is just slow to getting around to using the community. In this community's case, this was unfortunate, but an acceptable risk. We didn't remove accounts that had logged in, just ones that registered and never returned.
In another community, we kept everything that registered, unless it explicitly spammed the site. This method guarantees that users that registered will have an account if they return. It also increases the membership numbers so that the community looks larger.
Many forum software packages show number of registered users some place. It's much more impressive to see a number in the 10s-100s of thousands than it is to see a number in the thousands. Even if most of those are inactive, the number number gives the perception that the community is bigger.
This community was larger than the one where we deleted inactive users, so it was able to handle spam better on its own. That makes it hard to tell if if having these inactive accounts sticking around was a problem or not.
If the accounts are inactive, they aren't hurting you since they aren't being used. I liked deleting the inactive ones, because it reflected a much more accurate view of how large the community was.