The gemara says that without a mishna explicitly obligating kohanim to wear tefillin, we would have thought that they were exempt from the head tefillin while doing the avoda. The rationale is that since they cannot wear the arm tefillin while wearing bigdei kehuna because it will be a chatzitzah, they should also be exempt from the head tefillin, therefore we require a mishna to say that they aren't exempt from the head tefillin. It is clear from the gemara that while doing the avoda the kohanim were not able to wear the arm tefillin. There are techincally two options regarding the arm tefillin. Either to wear it under the bigdei kehuna which the gemara rejects due to the chatzitza or wear it over the bidgei kehuna which rashi and tosafos both reject because והיה לך לאות ולא לאחרים לאות, meaning that the tefillin shel yad must be covered and not exposed.
The Rashba in a Teshuva (תרכ"ז) permits one to wear tefillin even if there is something separating between the tefillin and his skin. He derives this from the fact that the only problem that rashi and tosafos mention with putting tefillin on top of the bigdei kehuna is that tefillin shel yad can't be exposed, it must be hidden. This implies that we aren't concerned for chatzitza and for tefillin shel rosh which should be exposed, there is not issue at all with placing them on top of something else. The Shulchan Aruch (27:5) relies on the Rashba for one who has a head injury to put the tefillin shel rosh on top of a bandage or thin cap (without a bracha). The M.B. (18) similarly permits one to put tefillin shel yad on top of a bandage (without a bracha) relying on the rashba when there is no alternative. However, under normal circumstances the Shulchan Aruch (4) writes that chatzitza is an issue and the tefillin must be placed directly on one's arm and head without anything blocking. The Rosh and Gr"a cite the source from על ידך and בין עיניך implying that it must be directly one one's skin without any chatzitzah.
Our gemara seems to support the position of the Rosh (against the Rashba) because the gemara questions why the tefillin shel rosh isn't a problem while wearing the hat, and responds that the hat sat higher on the head behind the tefillin. Why couldn't the gemara say that the tefillin was placed on top of the hat? It must be that the hat would be a chatzitza for the tefillin just as the tefillin would be a chatzitza for the hat. Perhaps this is why the Shulchan Aruch when relying on the Rashba insists on a "thin" hat, because he understands from the gemara that a thick had similar to the one worn by kohanim would indeed be a chatzitzah. However, it is hard to understand why a thin hat wouldn't be a chatzitza and thick one would, therefore the M.B. explains the problem with the thick hat to be that the tefillin will not be situated in the right place.
Another point that we can derive from this gemara is that kohanim are obligated to wear tefillin shel yad while doing avoda. The shita mikubetzes (6) asks that this seems to contradict a gemara in zevachim which exempts kohanim from the mitzvah of tefillin entirely while doing avoda based on עוסק במצוה פטור מן המצוה. The Shita explains that עוסק במצוה פטור מן המצוה only applies when he would have to put on tefillin at a time where otherwise he would be doing avoda so that the mitzvah would be a bitul of the avoda, but he is required to put on tefillin prior to the avoda and continue wearing them while doing the avoda since there is no contradiction. One of the commentaries in the back (Rashash Toibish) answers based on the Ran in Succah 25a who says that one is exempt from a mitzvah while performing another mitzvah even if he can do both. However, it would seem that since one can put on tefillin prior to beginning the avoda at a time when he hasn't even begun the avoda, even the Ran would require him to do so.