The gemara makes strange comparison. On the day which they imposed the 18 gezeiros, it says that it was קשה לישראל כיום שנעשה בו העגל, a difficult day for the Jews as the day they made the golden calf. In what way was it like the day of the golden calf? Rashi says that it is a continuation of the previous sentence, ואותו היום היה הלל כפוף ויושב לפני שמאי כאחד מן התלמידים, Hillel who was the Nassi was being mistreated and embarrassed into sitting in front of Shamai like one of the students. The humility of Hillel in allowing himself to be treated this way was a bizayon to Torah and therefore compared tot he day of the eigel. Following this approach the Minchas Bikurim on the Tosefta writes that just as the day of the eigel was a rejection of the Nasi, Moshe Rabbeinu, so too this day was a rejection of the Nasi, Hillel. Another possibility as to what Rashi means to say is that just as on the day of the Eigel, Aharon had a misplaced humility and didn't force them down, here too Hillel had a misplaced humility and allowed himself to be trampled by Shamai.
In the Tosefta 1:8 the statement of Hillel being forced to sit in front of Shamai is missing. In truth Tosafos 14b struggles with the idea that Hillel and Shamai were actually present on the day of the gezeiros of the 18 things - they were gezeiros of Beis Shamai and Beis Hillel, not Hillel and Shamai. According to the Tosefta that doesn't force us to say that Hillel himself was mistreated on that day, what is the comparison with the day of the eigel? The Minchas Bikurim on the Tosefta offers another explanation. Until that day, machlokes was very limited. There were only 3 arguments between Shamai and Hillel, but on that day machlokes became prevalent and widespread and it was like two toras were developing in the Jewish people. He is subtly referencing the Gemara in Yevamos 13b where one cannot act like Beis Shamai in a place of Beis Hillel and visa versa because it looks like שתי תורות.
Another suggestion as to why it was such a difficult day, I found in Lieberman's commentary on the Tosefta. He suggests that Torah needs to be decided based on a fair vote and majority. Generally speaking Hillel had the majority and Beis Shamai were the sharper ones (Yevamos 14a). However, on that day they happened to have a majority as Rashi comments on the Mishna 13b, and they took advantage of that majority by not letting anyone out and forcing a vote as rashi says on 17a. Torah needs to be decided by a fair אחרי רבים להטות, not a imposed and forced majority. Therefore, this event essentially undermined the Torah, just as the eigel undermined the Torah.
The Yefei Einayim cites a Tosafos in Gittin 36b that quotes a Yerushalmi that the students of Beis Shamai became so militant that they were killing the students of Hillel. That is why it was a terrible day, just like the day of the eigel where b'nei levi were forced to kill their brothers and sisters.