When observing an ostracism episode, observers may wish to know whether ostracism is justified or not. If ostracism appears unjustified, observers will likely blame the sources and sympathize with the target; if it appears justified, observers will likely blame and devalue the target. Here we introduce the “social dissimilarity rule,” which holds that observers base their moral judgments on dissimilarities between the members of the observed group. In five studies, participants either recalled observed ostracism episodes or observed group interactions in which one group member was ostracized (e.g., in a chat or a group-working task). Results show that if similar persons exclude a dissimilar target (target is an “odd-one-out”), observers attribute ostracism to malicious motives of the ostracizers, such as ingroup favoritism, and devalue the ostracizers. However, if ostracism cannot be explained by social dissimilarity between the sources and the target, observers assume that the target is being punished for a norm deviation (punitive motive) and devalue the target. Use of the social dissimilarity rule was neither moderated by cognitive load (Study 3) nor by the perceived essentiality of the group distinction (Study 4). But if participants knew that the target previously deviated from a norm, knowledge about the situation had a stronger effect on moral judgments (Study 5) than social dissimilarity. These findings further our understanding of how observers make moral judgments about ostracism, which is important given that an observer’s moral judgment can strongly impact bystander behavior and thus target recovery and well-being.