A 40-year-old male patient presented with a 3-week history of a painless ulcer on the dorsum of the tongue. He was a chronic smoker since the age of 15 years. The ulcer was well-defined, non-tender and indurated. An incisional biopsy was performed and the diagnosis of atypical histiocytic granuloma was made after histopathological examination. The lesion resolved spontaneously after the incisional biopsy and the patient was on follow-up for 6 months and no recurrences were observed. Here we report a rare case of atypical histiocytic granuloma. Hence, it must be borne in mind in the differential diagnosis of ulcers on the tongue.