Osteoblasts (OBs) are indispensable for the maintenance of hematopoietic stem cells (HSCs) in the bone marrow microenvironment. Here we investigated how Smad4 modulates HSC fate at distinct stages of OB development. For this, we conditionally knocked out Smad4 in cells expressing type I collagen (Col1a1) and osteocalcin (OC), respectively. Col1a1-expressing OBs were widely present in both the trabecular and cortical compartment, whereas OC-expressing OBs were predominantly located in the cortical compartment. HSCs from Col1a1 mutants displayed senescence-associated phenotypes. OC mutants did not exhibit HSC senescence-related phenotypes, but instead showed preferential HSC death. Of note, stromal cell-derived factor 1 expression was lower in Col1a1 mutants than control littermates, suggesting potential impairment of CXCR4-CXCL12-mediated HSC retention. Disruption of the CXCR4-CXCL12 axis by AMD3100 administration led to an increase in the senescence-associated β-galactosidase activity and low competitive potential. Collectively, our findings indicate that deletion of Smad4 in OBs differentially modulates HSC fate in a stage-dependent manner.