Tosafos quotes Rabbeinu Tam that the takanas ezra of chafifa is only on the hair, not on the rest of the body. Really it is a machlokes rishonim (see my sefere mayim rabim 287), but m'ikar hadin we paskin like rabbeinu tam. One of the proofs of Rabbeinu Tam is the gemara in Nida שתהא אשה מדיחה בית קמטיה במים, from which Tosafos deduces that only the folds in her body need to be washed, not the rest of her body. However, the sidrei tahara (199:9 - quoted in my sefere 288) says that according to the maharik who says that when the gemara says בית קמטיה במים, it means water to the exclusion of other liquids, Tosafos has no proof that the rest of the body doesn't need to be washed - maybe it needs to be washed just that בית קמטיה is special in that we insist on water, but the rest of her body although it needs to be washed, it can be washed with other liquids.
Also, the Cheshek Shlomo (R' Shlomo Vilna) quotes the sefer hateruma who asks why the takana of chafifa was only for women and not for men who are toveling for taharos? The sefer Hateruma says based on a yerushalmi in pesachim that since men are frequently going to the mikva for taharos, we didn't require chafifa for them. The cheshek shlomo asks that the yerushalmi implies that only kohanim don't need chafifa, but non kohanim would. Cheshek shlomo suggests that perhaps the chatziztah that we are concerned about is only a mi'ut that one is makpid about, which women tend to be makpid about, not men. I don't understand why he doesn't simply say that men tend to have short hair whereas women tend to have long hair, so chafifa is more necessary for women (maybe the whole long hair for women or short hair for men is a modern style!). But, in truth it seems that the cheshek shlomo forgot a tosafos in chulin 10a who asks on a mishna which is pashtus speaking about a man going to the mikva (it is all b'lashon zachor), from chafifa. Clearly Tosafos implies that even men require chafifa. In fact, Tosafos at the very end quotes the same yerushalmi in pesachim as a source that only kohanim don't need chafifa ans suggests that the mishna was speaking about kohanin, which would imply that men who aren't kohanim would need chafifa for taharos.