The gemara says that even tzadikim are held accountable for the sins of the wicked if they fail to give them rebuke. The failure to rebuke demotes the tzadikim gemurim to the level of regular tzadikim who are not protected from tragedy, rather they are placed at the front of the line - ממקודשי תחלו. The Rama writes in Hilchos Yom Kippur (608:2) that 3 conditions have to be met to exempt a person from rebuking his friend: 1. They are violating accidentally. 2. They will continue to violate even if they are rebuked. 3. The prohibition being violated isn't explicit in the Torah (because something that is not explicit in the Torah they don't believe the rebuker that it is in fact assur, whereas something that is explicit in the Torah they presumably are familiar with the prohibition and therefore are not considered accidental violators - M.B.). However, if any of these conditions aren't met, there is an obligation to rebuke an individual until he hits or curses you, and a community at least once.
In Yechezkel 9 when the l'vush habadim is told to draw a mark on the heads of the tzadikim to protect them from the "mashchisim" that will be entering the city, the pasuk says that the destruction began with the "holy ones". The gemara explains that they should have protested. The aveiros that led to the destruction included avoda zara and many other things that are explicit in the Torah so the tzadikim should have rebuked them. Tosafos asks from the concept of מוטב שיהיו שוגגין ואל יהיו מזידין and answers that it only applies when you are certain that they won't listen. Tosafos seems to understand that if the tzadikim in the generation were absolutely certain that their rebuke would be ignored, they wouldn't be obligated to rebuke. Although the type of sins we are speaking about are sins that are explicit in the Torah, and being violated intentionally, they would not be responsible to rebuke if they knew that the violators wouldn't listen - Tosafos disagrees with the first and third condition of the Rama. Tosafos holds that so long as the second condition is met, that they won't listen, there is no obligation to rebuke.This point is already articulated by the Biur Halacha who cites the sugya in Shabbos 55 speaking in the very same context and indicates that if one is certain that he won't be listened to, he wouldn't have to rebuke even on something explicit in the Torah. However, the biur halacha suggests that this may only help to alleviate the punishment, but the mitzvah of rebuke would still be there even if one was sure that he would be ignored. But, the opinion of the Yerei'im (and Tosafos in Shabbos 55) is that when you are sure that they will not listen, even if they are violating intentionally and it is explicit in the Torah, there is no obligation to rebuke.
The Biur Halacha concludes that although there is a mitzvah to rebuke an intentional violator who will not listen, that is only if they are still for the most part religious people. There is no obligation to rebuke those who are completely void of Torah and pay no attention to any mitzvos because they are no longer considered "עמיתך". Based on this approach, Tosafos in our sugya fits with all the conditions of the Rama because the violators in Yechezkel wouldn't qualify as עמיתך so there wouldn't be an obligation to rebuke if it was clear that they wouldnt listen.
The Birkei Yosef (cited in Biur Halacha) explains that the condition of the Rama that it not be something explicit in the Torah only applies when we have the ability to forcefully implement a stop to the behavior (even if the violators don't have a change of heart) but when it won't accomplish anything, there is no mitzvah at all.