This study aims to provide further understanding of physiologic and symptomatic changes and radiographic abnormalities due to exposure to silica, asbestos, and coal dusts. Questionnaires and pulmonary function tests were given to 220 silica, 277 asbestos, and 511 coal workers from three different industries in China. Posteroanterior chest radiographs were classified as stages 0, I, II, and III according to degree of parenchymal fibrosis. Significantly poorer pulmonary function and a higher prevalence of dyspnea and chronic cough were observed in workers with pneumoconiosis than those without, irrespective of dust type. Workers with stages II and III silicosis had worse pulmonary function and more common symptoms relative to workers with equivalent coal workers’ pneumoconiosis or asbestosis. After adjusting for relevant confounders, reductions in the spirometric parameters and single breath diffusing capacity for carbon monoxide (DLco) and the occurrence of respiratory symptoms were associated with increasing stage of silicosis, whereas lower DLco and the occurrence of symptoms were associated with increasing stage of asbestosis and coal workers’ pneumoconiosis. The study suggests that despite the differences in degree and pattern due to exposure to different fibrogenic dusts, respiratory impairments of all of the workers are associated with the presence and progression of parenchymal fibrosis and smoking.