You raise a good question. Clinical Psychologist, Seth Meyers wrote a blog response in 2012 entitled: During Sex, Is It Okay to Fantasize About Someone Else? Why? Fantasies can be harmless or hurtful. He gives a great rabbinic answer: “Yes and No.” He believes that there are natural instincts for someone to see others as attractive and even have affection for them. He also argues that no one individual can meet all of our needs -- all of the time. He encourages partners in the couple to maintain their individuality and for the partner to give the other a certain freedom to be the person they need to be.
However he addresses the question you ask with greater detail. Rightly so, he believes there is no partner who ever wants to be in the middle of an intimate moment and wonder if their partner is thinking about someone else. While members of a couple may have a fantasy about another person, it is not natural to do so in the midst of the act with your partner. In fact, if it is happening more often, then you may be using it as a coping mechanism for handling your feelings about your relationship. You should examine what is bothering you about the other person. Are you bored of the relationship, are you angry or disturbed by something your partner had done? In fact, instead of giving yourself permission to think about others, you may want to explore what is wrong in your relationship.
Both Dr. Ruth Westheimer and Rabbi Shmuely Boteach have passages in their books that while fantasies may happen, we should be working on strengthening our relationship with our partner. Our Sages took into account that human beings are built for procreation and pleasure. Both are important in the relationship going so far as to indicate that a man’s responsibility is to ensure the pleasure of his spouse and that both should bring a sense of modesty and sacredness into the relationship. Doing so helps to bring passion and energy to what is a most intimate act.
One of my favorite teachings is about the Hebrew words for man and woman – Eesh and Eesha. If you remove one letter from each word, you are left with the word for fire. Fire can be a synonym for passion. The letters removed are “yud” and “hay” which together spell a name of God. If the Divine Presence is with a man and a woman, all is well, but without that Divine Presence, the fire becomes destructive in nature. So my answer is that while fantasies may happen, it is important to work on your relationship with your spouse so at the most intimate of moments, you are both in that sacred moment giving to each other the needed affection, love and passion that should exist between spouse.