A case–control study was conducted to estimate the association of cartilage oligomeric matrix protein (COMP) with knee osteoarthritis (OA) and to examine the potential utility of COMP as a diagnostic and prognostic biomarker in early knee OA. The COMP levels were estimated in the blood sera of 150 subjects belonging to study group (n = 100) and control one (n = 50). Patients with confirmed clinical isolated knee OA diagnosed through American College of Rheumatology criteria were included and were without any other cause of knee pain. ELISA was used to determine the levels of COMP, interleukin-1β (IL-1β) and tumor necrosis factor-α (TNF-α). The median (range) serum COMP levels were observed to be 1117.21 ng/ml (125.03–4209.75 ng/ml) in OA patients and 338.62 ng/ml (118–589 ng/ml) in control subjects with p < 0.001. The COMP levels of study group were negatively correlated (correlation factor −0.88) with disease duration and positively correlated with age, BMI, pain score and IL-1β with correlation factors 0.86, 0.63, 0.76, and 0.79, respectively with p < 0.001. Gender differentiation was found in study group with 52% higher COMP level in males as compared to that of females. There was no significant correlation of COMP levels with radiological grading, erythrocyte sedimentation rate (ESR), hemoglobin (Hb), and TNF-α. The serum COMP levels may be used as a diagnostic OA marker along with prognostic value in determining the patients at risk of rapidly progressing this debilitating joint disease. The serum COMP level remains significantly high in first 3 years of disease duration.