The gemara learns out that one is obligated to continuously touch their tefillin while wearing them to avoid a "hesech ha'da'as", from the tzitz. The gemara seems to make a real kal v'chomer and regards this as a din d'oraysa. The Shita (5) asks, how then is one allowed to daven or read shema while wearing tefillin, since by concentrating on the davening they will inevitably not be concentrating on the tefillin? He answers that in truth it is a violation of hesech ha'da'as from tefillin, but it is permitted based on the concept of עוסק במצוה פטור מן המצוה. Since he is involved in the mitzvah of concentrating on the davening, he is exempt from the mitzvah to concentrate on the tefillin. The application of עוסק במצוה פטור מן המצוה assumes that the prohibition of having a hesech ha'da'as from tefillin is an outgrowth of a positive mitzvah to concentrate on the tefillin, and is therefore regarded as a mitzvah rather than an issur. The concept of עוסק במצוה פטור מן המצוה would only entitle one to forgo a mitzvah, but not to violate an issur.
The Biur Halacha (44:1) quotes the pri megadim who says that even during davening one's mind should be on the tefillin and not entirely focused on the davening. However, the biur halacha says that the Ari z"l always had his mind on his tefillin even during davening, except during shemoneh esrei and learning Torah, implying that during shemoneh esrei one doesn't need to have their mind on the tefillin. The Biur Halacha quotes a fascinating Ohr Zarua -
וצריך אדם כשהוא לבוש תפילין לזכור שתפילין עליו ולא להסיח דעתו מהם, שלא יישן בהם ולא יפיח בהם, כי מחמת שזוכר שנושא שם הנכבד עליו אינו בא לידי קלות ראש ותופס יראת שמים בלבו וכו' הלכך כשאדם מכוין לבו בתפלתו או בהלכה שלו אין זה היסח הדעת דכל שכן דאיכא יראת השם
The Ohr Zaruah holds that the ikar mitzvah is not to be focused on the tefillin but rather to be focused on yir'as hashem. So long as a person is involved in service of Hashem, such as learning and davening, it isn't considered a hesech ha'da'as even though his mind is not at all on the tefillin. Based on this we don't need the concept of עוסק במצוה to justify how one can concentrate on their davening while wearing tefillin.
In truth, the violation of hesech ha'da'as isn't just a lack of concentration, but would only be violated if one is actively involved in light headed behavior. The Shulchan Aruch (o.c. 44:1) paskens that one may sleep a temporary sleep in their tefillin. The Mishna Berura (3) struggles with the issue of hesech ha'da'as and explains that the violation of hesech ha'da'as is only if one is involved in frivolous or silly behavior, but the absence of concentration on the tefillin doesn't constitute a hesech hada'as. When one is sleeping and has nothing whatsoever on their mind, it isn't considered a hesech hada'as. This approach seems to contradict our gemara. Why would one have to constantly touch tefillin to avoid a hesech hada'as, if the issue of hesech hada'as cannot be violated by a passive unawareness of the tefillin? The M.B. himself deals with this question and explains that we learn from the tzitz that it is forbidden to have a hesech ha'da'as, but the Rabbonon instituted as an ideal to constantly touch their tefillin to avoid one's mind from wandering and leading to improper thoughts. However, the violation is only if one would be actively thinking about other thoughts.