(I'm going to treat this is a non-SE comment question...)

First of all, a quick note should be added after you clean them up. It might say something like this:

Moderator's note: please do not add comments that are not constructive or comments meant to be funny. If you want to talk about this question use [other means of communication] to do so.

After that, you can't do much. Users will do whatever they want, and it is pointless to ban someone for adding a comment. You can't fight this...

Specific to SE, moderators can either lock comments or they can protect the question (requres 10 rep) to prevent these comments.

(I'm going to treat this is a non-SE comment question...)

First of all, a quick note should be added after you clean them up. It might say something like this:

Moderator's note: please do not add comments that are not constructive or comments meant to be funny. If you want to talk about this question use [other means of communication] to do so.

After that, you can't do much. Users will do whatever they want, and it is pointless to ban someone for adding a comment. You can't fight this...

Specific to SE, moderators can lock comments to prevent this.