Do interlocutors interpret language as though they were “in the shoes of” the speaker? People can interpret utterances describing actions from an internal perspective (as though they are performing the action), or from an external perspective (as though they are observing the action). In Experiment 1, the speaker produced sentences such as I am cutting the tomato, and the addressee matched these sentences against pictures taking internal or external perspectives to the action. The addressee tended to take an external perspective on sentences involving I and an internal perspective to sentences involving you, irrespective of whether the interlocutors were adjacent to or opposite each other. In Experiment 2, the interlocutors alternated between acting as speaker and addressee, and the addressee tended to take an internal perspective for sentences involving I and an external perspective for sentences involving you; we also assessed the perspective that the speaker adopted. The results argue against a simple simulation account, and also against an account in which the comprehender adopts a perspective that straightforwardly accords with the meaning of the sentence. Instead, the comprehender appeared to take a complementary perspective to the speaker, perhaps to retain independence from the speaker.