I would suggest that in a site where a large number of users will be seeing the content, you don't want a set number, but rather a set percentage.
If you use a fixed number and set the number high, less popular articles will have a hard time getting rid of undesirable content. If you set it too low, people who simply don't like an opinionated comment may remove it even though they are a small minority of the viewers. That said, you also want a minimum number of required flags as well. Otherwise, if the first viewer doesn't like it and flags it, then 100% of viewers flagged it.
As for exact levels to set at, this is just as tricky as anything in dealing with user moderated content. It depends very widely on your readers and how pro-active they are about reporting content. Just like a good spam filter, it takes time and effort to train properly for your environment.
If it isn't too difficult to do so, I'd recommend putting the system up for a week or two where reports simply send it to a queue of comments to be manually reviewed. Take a look at what kind of report rates you are getting for both accurately and inaccurately flagged comments and then set the levels such that you have relatively few false positives while generating few or no false negatives.
If that simply isn't an option, you'll need to guess and check. I'd start with a minimum of flags from about 0.5% of the typical day 1 viewership of a small article and make sure at least 1% to 2% of the viewers have flagged, but that's probably still going to make some false positives unless you have a very well behaved viewership, so be sure to check what posts are getting removed and raise it as necessary.
If you would prefer early false negatives, I'd try something more like 1.5-3% of day 1 viewership as a min and 5% of viewers have flagged. That's a lot more likely to result in false negatives due to the portion of people that typically bother reporting anything but the worst violations, but it should be a lot safer against false positives. Still, check for errors in either direction though, as individual communities can differ greatly.
Another, more advanced option, is to have allow a pool of active participants who show good judgement in comment moderation to review flagged posts as well. This is actually similar to how Stack Exchange deals with questions and answers that don't fit. You gain the ability to access a review queue after a certain level of involvement and then have the ability to actually review if issues are actually issues. It is likely a more accurate system, but it is also much harder to get running well.