Plant–herbivore dynamics is a major topic in ecological research, but empirical knowledge on the ecological effects of different densities of large grazers from fully replicated experiments is rare. Previous studies have focused on grazing vs no grazing, and our understanding of the extent to which different levels of grazing alter vegetation composition, and how quickly such effects can be measured, is therefore limited. We performed a fully replicated, short-term (four-year) experiment using large enclosures (each ∼0.3 km2) with three different sheep densities (no grazing, low grazing and high grazing, respectively) in an alpine environment with summer grazing in southern Norway to address these issues. Sheep grazing mainly affected plant species at high densities of sheep as compared to no sheep after a four-year treatment; few effects of low sheep densities were detectable. Highly selected herbs, herbs suggested vulnerable to trampling, and woody species decreased, while most graminoids, one ruderal, one prostrate species and two bryophyte taxa increased at high sheep densities. We found contrasting responses within main functional groups highlighting that fine details of plant life histories need to be known for responses to grazing to be successfully predicted. Vascular plant cover and bare soil responded to sheep density after two years of treatment, but only for one of the species was frequency change observed at this stage. Overall, plants in low grazing plots were found to be almost unaffected. Changes in abundance were mainly found at the no grazing and high grazing treatments. Plant species that decreased at high grazing generally increased at no grazing and vice versa, suggesting a response to both cessation of grazing and enhanced grazing respectively. Our study demonstrates, beyond a simple comparison of heavily grazed and non-grazed sites, that herbivore effects on plants are typically non-linearly related to herbivore density, and that the speed of plant responses will depend both on the plant property examined and the grazing pressure.