We investigate the role of steric interactions in defining side-chain conformations in protein cores. Previously, we explored the strengths and limitations of hard-sphere dipeptide models in defining sterically allowed side-chain conformations and recapitulating key features of the side-chain dihedral angle distributions observed in high-resolution protein structures. Here, we show that modeling residues in the context of a particular protein environment, with both intra- and inter-residue steric interactions, is sufficient to specify which of the allowed side-chain conformations is adopted. This model predicts 97% of the side-chain conformations of Leu, Ile, Val, Phe, Tyr, Trp and Thr core residues to within 20°. Although the hard-sphere dipeptide model predicts the observed side-chain dihedral angle distributions for both Thr and Ser, the model including the protein environment predicts side-chain conformations to within 20° for only 60% of core Ser residues. Thus, this approach can identify the amino acids for which hard-sphere interactions alone are sufficient and those for which additional interactions are necessary to accurately predict side-chain conformations in protein cores. We also show that our approach can predict alternate side-chain conformations of core residues, which are supported by the observed electron density.