Tumor budding is a major risk factor for T1 colorectal cancer. Quality control of the pathological diagnosis of budding is crucial, irrespective of the pathologist's experience. This study examines the interobserver variability according to pathologists' experience and evaluates the influence of cytokeratin (CK) immunostaining in the assessment of budding. Hematoxylin-eosin (HE) and CK-immunostained slides of 40 cases with T1 primary colorectal cancer were examined. Budding grades were individually evaluated by 12 pathologists who we categorized into three groups by their experience (expert, with >10 years of experience (n = 4), senior, with 5–10 years (n = 4), and junior, < 5 years (n = 4)). The results revealed a tendency for the more experienced pathologists to assign higher budding grades compared to the less-experienced pathologists. In the junior group, the interobserver variability obtained with HE slides was poor, but it was markedly improved in the evaluation using CK-immunostained slides. The benefit of CK immunostaining was only slight in the expert group. CK immunostaining would be useful when a pathologist is not experienced enough or does not have enough confidence in the assessment of budding.