A-55-year-old man with a 2-year history of left proptosis with painless swelling of the upper and lateral bulbar conjunctiva was referred. He had developed diplopia in left gaze. Orbital CT showed left proptosis with a mass measuring 2 × 1 cm in the superolateral and lateral left orbit, with lateral rectus muscle infiltration. The lesion was excised and was found to be diffuse, and an infiltrative mass affecting the anterior portion of the lateral rectus muscle was also removed. The histopathologic diagnosis was pleomorphic lipoma. Only 7 cases of pleomorphic lipomas occurring in ocular adnexal tissues or in the orbit have been previously reported, but in none of the cases had an infiltration of the lateral rectus muscle or diplopia been described before. The histopathologic features and differential diagnosis of this type of soft tissue tumor are also described.