The objective of the study has been to examine whether the freestyle dressage classes are judged reliably and to elaborate a method assessing the consistency of judging. The data contained 13 000 marks of the ten best-ranked horses at nine Grand Prix classes and four Intermediate I classes from ten international competitions. The marks concerned 119 horses in total and were given by 37 judges. A method for evaluating the consistency of judging has been proposed. The index of disagreement (ID) assesses the disagreement of ranking by an individual judge relative to the general ranking based on the sum of marks awarded by five judges. The results show that the mean ID of individual judges is highly differentiated. The ID is influenced by the judge's position in the arena. The consistency of judging was lower in Intermediate I Freestyle Tests than in Grand Prix Freestyle Tests and it varied in different competitions. The conclusion of the study is that the results of the freestyle dressage classes are often biased. The present system of judging should be permanently checked. The offered method of evaluating agreement between judges' rankings may help to improve the consistency of judging and the reliability of the horses' scores.