The present study sought to determine whether contextual information available when viewing social interactions from third-person perspectives may influence observers’ perception of the interactants’ facial emotion. Observers judged whether the expression of a target face was happy or fearful, in the presence of a happy, aggressive, or neutral interactant. In 2 experiments, the same target expressions were judged to be happier when presented in the context of a happy interactant than when interacting with a neutral or aggressive partner. We failed to show that the target expression was judged as more fearful when interacting with an aggressive partner. Importantly, observers’ perception of the target expression was not modulated by the emotion of the context interactant when the interactants were presented back-to-back, suggesting that the bias depends on the presence of an intact interaction arrangement. These results provide valuable insight into how social contextual effects shape our perception of facial emotion.