Osteogenesis is a complex process which relies on the coordination of signals and transcription factors. BMP-2/Smad5 signal transduction pathway plays an important role in the process. Recent evidence indicates that microRNAs (miRNAs) act as important post-transcriptional regulators in a large number of biological processes including osteoblast differentiation. In this study, we investigated the effect of miR-106b-5p and miR-17-5p on osteogenic differentiation. We observed an obvious decreasement in miR-106b-5p and miR-17-5p levels during osteogenic differentiation. By using gain- and loss-of function experiments, we noticed that miR-106b-5p and miR-17-5p could suppress the osteogenic differentiation of C2C12 and MC3T3-E1 cells. In addition, we performed bioinformatic analysis, western blot and luciferase reporter assay to confirm that miR-106b-5p and miR-17-5p could regulate Smad5 expression negatively. When we inhibited Smad5 expression by siRNA, the effects of miR-106b-5p and miR-17-5p inhibition on osteogenesis of C2C12 cells could be significantly reversed by Smad5 RNA interference. Furthermore, silencing of miR-106b-5p and miR-17-5p in sham and ovariectomized (OVX) mice increased bone formation and bone mass, resulting in an improvement of trabecular microarchitecture. Taken together, these data showed that miR-106b-5p and miR-17-5p are novel Smad5 regulators, and they have a crucially physiological function in bone formation and osteoblast differentiation.