Molecular dynamics (MD) simulations have become a central tool for investigating various biophysical questions with atomistic detail. While many different proxies are used to qualify MD force fields, most are based on largely structural parameters such as the root mean square deviation from experimental coordinates or nuclear magnetic resonance (NMR) chemical shifts and residual dipolar couplings. NMR derived Lipari–Szabo squared generalized order parameter (O2) values of amide N—H bond vectors of the polypeptide chain were also often employed for refinement and validation. However, with a few exceptions, side chain methyl symmetry axis order parameters have not been incorporated into experimental reference sets. Using a test set of five diverse proteins, the performance of several force fields implemented in the NAMDD simulation package was examined. It was found that simulations employing explicit water implemented using the TIP3 model generally performed significantly better than those using implicit water in reproducing experimental methyl symmetry axisO2 values. Overall the CHARMM27 force field performs nominally better than two implementations of the Amber force field. It appeared that recent quantum mechanics modifications to side chain torsional angles of leucine and isoleucine in the Amber force field have significantly hindered proper motional modeling for these residues. There remained significant room for improvement as even the best correlations of experimental and simulated methyl group Lipari–Szabo generalized order parameters fall below anR2 of 0.8.