We present an example when profile measurement and modeling of an Elekta Agility multileaf collimator (MLC) had a large effect specifically on arc therapy plan quality assurance (QA) results using ArcCheck. ArcCheck absolute dose measurements of these plans were systematically lower than planned by 3–10%. Failing QA results were seen even with unmodulated static and conformal arcs. Furthermore, the effect was found to be dependent on collimator angle, with worse results associated with near-zero collimator angles. In contrast, step-and-shoot QA results were not affected. Changing the beam model to match steeper profile measurements obtained using a different measurement device resolved the problem. This case study demonstrates that conventional gamma index analysis can be sensitive to small profile modeling changes.