Disclaimer: The following answer is the result of a few days' investigation by a non-expert in the field, drawing entirely on publicly available sources. It should not be taken as a comprehensive review of the literature.
This question piqued my interest but I don't know of any data or studies that address the very specific comparison you're asking about. There's quite a lot of research out there on web communities but those that look at empirical data tend to focus on one individual platform. I saw a fair number of articles that drew entirely from Usenet, which isn't totally irrelevant but doesn't capture the range of what a modern web forum provides.
The right technology for you
One article from the University of Haifa seems to argue that the compatibility between a particular technology and its user base is more important than the choice of technology itself. The authors say that a taxonomy of "virtual publics" should address: the subject of conversation; the geographic and demographic characteristics of the user base; the social structure of the community; and the discourse architecture, which is essentially the term they use to refer to the platform and its underlying technology.
To achieve [their] goals the managers of virtual publics need to
ensure that the virtual public's discourse architecture
enables/encourages the discourse form desired. (218)
The article points to one main cost of participation in "virtual publics" being the stress users experience as a result of a communication-processing load. The more content they have to read and process in order to participate, the greater the stress. Too much stress leads to information overload, but the consequence of which is only vaguely stated; "beyond a particular communication-processing load, the behavioral stress zones encountered will make group communication unsustainable" (218-9).
The claim here is not that technology will determine online behavior,
rather, that technology is both an enabler and constrainer of actions.
Each class of [computer-mediated communication tools] will have its
own associated stress zones that can be measured. For each class of
CMC-tools, it is also likely that the point at which information
overload becomes an issue will relate to the nature of the virtual
public's discourse. For example, virtual publics which aim to support
empathetic communities ... may require a greater level of
interactivity and thread depth to sustain appropriate discourse than a
virtual public focused on software support. (220)
The practical response is to ask, well, what's the consequence of information overload? Does it tend to cause users to participate less, or to disengage entirely? Is it simply a pressure that imposes a carrying capacity on the community, or is it something catastrophic that could cause a community to collapse? These questions are raised in a follow-up paper from the same researchers:
Individuals can take a range of actions to reduce the impact of
information overload resulting from group CMC.... [which] can be
combined into two primary options for a population of experienced
users who cannot easily adopt new information management techniques.
The first option is simply to end participation. The second option
is to change ongoing communicative behavior.
The main point I take away is that either type of platform could be better at engaging and retaining users, depending on the nature of the user base and the goals of the community. This seems to fit with what I see in web communities in that certain demographics are much more comfortable with certain technologies, especially with respect to age.
The other key point is that information overload reduces engagement and retention. So in choosing between platforms, ask yourself how each option might allow your users to manage the overall volume of information they would need to process in order to participate a lot or a little.
The authors of the second paper conclude by saying that they "expect to see more work on large-scale comparative analysis of virtual public discourse dynamics for a variety of technologies (Listserv, IRC, Web-based bulletin board systems, etc.)" (206) but I haven't come across a study yet that specifically compares mailing lists to other platforms.
Mailing list case studies
One case study of a mailing list centered around a particular open-source software development project concluded that the frequency with which postings to the list went unanswered was a significant source of frustration among participants. You might consider whether this is a problem for your community, and whether it would be more or less of an issue on a web forum based on your familiarity with the user base.
Another mailing list case study observed that as the community grew, messages broadcast to the list for the sole purpose of disseminating information increased in proportion, compared to messages that were part of interactive discussions. The authors conclude by suggesting that, "if the overall trend toward list usage with low levels of interactivity continues, it may in the long term reduce members' motivation to use [the list] for serious discussions" (322). They also suggest, as potential improvements, splitting the list into sub-lists to address different uses; providing a mechanism for users to filter which categories of message they receive; and augmenting the list with "new and more effective communication media such as social networking tools, wikis, and blogs" (324). If your mailing list community is similar to the one addressed by that case study, you might consider exploring those suggestions before migrating entirely to another platform.
In case of opinion, quote someone else
Finally, though it's not of the same rigor as a journal article, I did come across a blog post that talks specifically about the pressure for mailing lists to move over to web forums:
I think the discussion has come up at least twice on every mailing
list I'm on: Why don't we move this discussion to a web-based forum?
I got to thinking about it -- why do I, and so many other competent,
technical folks prefer the archaic mailing list format to an online forum?
The answer is simple: quantity. I think that those of us who prefer
mailing lists simply participate in more different communities.
It would be gratuitous to quote the entire post here, so I'll summarize. The author sees web forums as ideal for welcoming new users and as a space "where participants invest a lot of time on a daily basis." The down-side in his view is that web forums tend to demand this greater investment of time because there are so many different types of forum software to learn, as opposed to mailing lists which all come to the same inbox.
He also points out that that lower barriers to entry result in a lower signal-to-noise ratio, and the public nature of posts attracts much more spam. Dealing with these certainly requires more leadership and moderation activity than some communities are able to provide. (That being said, I've seen these roles transform users into more productive community members, in a way that I think was a net gain for the community.)
...in spite of all the flaws of mailing lists, it's much easier for
people to participate in many communities at once when they all come
to one place -- email.
You will find lots of differences of opinion, but in the open source
world, a huge number of competent developers hang out on mailing
lists, and simply do not have time to participate in forums. If you
want to reach them, you go to where they are. And lots of other
communities grow under similar circumstances.
My conclusion in turn is that it's part of a community administrator's job to be familiar enough with a community to make an informed decision about which platform is going to ultimately be best for the community. What's "best" should be defined in terms of specific outcomes, and trade-offs have to be identified. Your goal might be to increase the size of your user base, but that could conflict with a goal of retaining current users, especially if there's a difficult transition between platforms or the user base increases too rapidly. If the research shows anything, it's that there's no one right answer.
One thing to add: It's great that this discussion took place openly within your community. There are cases when founders or owners take matters into their own hands and decide what's best for a community on their own. I wouldn't say that never works, but it certainly doesn't hurt to test the waters a bit and get some user feedback about proposed changes.
Jones, Q., & Rafaeli, S. (2000). Time to split, virtually: 'Discourse architecture' and 'community building' create vibrant virtual publics. Electronic Markets, 10(4), 214-223.
Jones, Q., Ravid, G., & Rafaeli, S. (2004). Information overload and the message dynamics of online interaction spaces: A theoretical model and empirical exploration. Information systems research, 15(2), 194-210.
Toral, S. L., Martínez-Torres, M. R., & Barrero, F. (2009). Modelling mailing list behaviour in open source projects: the case of ARM embedded Linux. Journal of Universal Computer Science, 15(3), 648-664.
Yu, M. Y., Lang, K. R., & Kumar, N. (2010). Supporting better communication in academic communities of practice: An empirical study of AIS/ISWORLD. Communications of the Association for Information Systems, 26(1), 16.
Locke, J. (2010, March 10). Mailing list or forum? A theory... [Blog post].