The gemara says that when one calls someone a rasha they are allowed to be יורד עמו לחייו. Rashi in Baba Metzia 71a explains that it is not a justified right, rather a predictable outcome that it will result in creating serious friction between them - they will fight to the point where it is as if he is coming to kill him. Rashi also mentions another possibility, that it is in fact a right to compete against him for the purpose of causing him a financial loss. However, Rashi questions this approach - וקשה בעיני שיתירו חכמים לישראל להנקם ולגמול רעה - it cannot be that the chachamim sanction revenge. Yet, Rashi in kiddushin quotes only the second approach that the victim is allowed to hate the name caller and even do things that negatively effect his parnasa. To deal with Rashi's question in Baba metziah that this does not seem to be the way of the chachamim to sanction revenge, Rashi adds that Beis din is not required to get involved and prevent the victim from causing a financial loss. Although they may not encourage it, it is allowed. Tosafos in Baba Metziah quotes from teshuvos ha'geonim that you are permitted to directly destroy 1/3 of his produce, and then questions where such a right would come from.
Tosafos explains that both the calling someone an eved which results in niduy, and the calling him a mamzer which results in malkus, are in fact mida k'neged mida (a measure for measure). Calling him an eved who descends from k'na'an and is damned forever, is also cursed by being placed in niduy. Calling someone a mamzer is claiming that the person deserves malkus for violating the prohibition of mamzer marrying a bas yisroel, and therefore is punished with malkus (Tosafos implies that this is only if the victim is in fact married - see ritva). Tosafos fails to explain the mida k'neged mida for calling someone a rasha - why are you entitled to ruin him financially?
The Tosafos R"i Hazaken explains: כלומר, אין ב"ד נזקקין לו ראלא מותר לו לצערו או ליטול אומנתו ממנו וכו' הואיל ומוציא עליו שם שהוא רשע, וגורם לו בשם זה שלא יתנו לו שום דבר להרויח ויצערוהו בכל יכולתם שיש לבני אדם הכשרים להבדל מן הרשעים בשם עסק ולצער אותם כפי היכולת, משום הכי יורד עמו לחייו
The Tosafos R"i Hazaken seems to understand differently then Rashi (he later quotes rashi and says that he prefers his own approach). Rashi understood that the victim is allowed to ruin the livlehood of the person who called him a rasha out of hate, merely for the purpose for ruining him. The Tosafos R"i Hazaken understands that the result of calling the victim a rasha will be that he will have trouble earning a living since people will not want to have what to do with him, so chazal give him the right to take the livelihood of the one who caused him these problems. Perhaps the victim can't ruin the parnasa fo the one who called him a name just for the purpose of ruining him, but beis din won't stop him from taking the parnasa of the other in order to support himself. If this is true that the victim can recover his loss at the expense of the other, but cannot intentionally ruin him for fun, it would answer the question that Rashi asks in baba metziah, how chazal can allow for revenge. The answer is that they don't, they merely allow the victim to recover his losses (but the tosafos R"i hazaken seems to allow even causing unnecessary pain to the other, which implies that it is permitted even though it will not provide benefit to the victim).