Quadriceps weakness has been associated with knee osteoarthritis (OA). High-frequency localized muscle vibration (LMV) has been proposed recently for quadriceps strengthening in patients with knee OA. The purpose of this study was (a) to investigate the clinical effectiveness of high-frequency LMV on quadriceps muscle in patients with knee OA and (b) to disentangle, by means of surface electromyography (sEMG), the underlying mechanism. Thirty patients, aged between 40 and 65 years, and clinically diagnosed with knee OA were included in this randomized, controlled, single-blinded pilot study. Participants were randomly assigned to two groups: a study group treated with LMV, specifically set for muscle strengthening (150 Hz), by means of a commercial device VIBRA, and a control group treated with neuromuscular electrical stimulation. Clinical outcome was measured using the Western Ontario and McMaster Universities Osteoarthritis Index, Visual Analogue Scale, knee range of motion, Timed Up and Go test, and Stair climbing test. To assess changes in muscle activation and fatigue a subgroup of 20 patients was studied with the use of sEMG during a sustained isometric contraction. The LMV group showed a significant change in Western Ontario and McMaster Universities Osteoarthritis Index score, Visual Analogue Scale score, Timed Up and Go test, Stair Climbing Test, and knee flexion. These improvements were not significant in patients treated with neuromuscular electrical stimulation. sEMG analysis suggested an increased involvement of type II muscle fibers in the group treated with LMV. In conclusion, the present study supports the effectiveness of local vibration in muscle function and clinical improvement of patients with knee OA.