The goal of the presented study was to compare the biocompatibility and cellular responses to porous silk fibroin (SF) scaffolds produced in a water-based (UPW) or a solvent based process (HFIP) using two different SF sources. For that reason, four different SF scaffolds were implanted (n = 6) into drill hole defects in the cancellous bone of the sheep tibia and humerus. The scaffolds were evaluated histologically for biocompatibility, cell–material interaction, and cellular ingrowth. New bone formation was observed macroscopically and histologically at 8 weeks after implantation. For semiquantitative evaluation, the investigated parameters were scored and statistically analyzed (factorial ANOVA). All implants showed good biocompatibility as evident by low infiltration of inflammatory cells and the absent encapsulation of the scaffolds in connective tissue. Multinuclear foreign body giant cells (MFGCs) and macrophages were present in all parts of the scaffold at the material surface and actively degrading the SF material. Cell ingrowth and vascularization were uniform across the scaffold. However, in HFIP scaffolds, local regions of void pores were present throughout the scaffold, probably due to the low pore interconnectivity in this scaffold type in contrast to UPW scaffolds. The amount of newly formed bone was very low in both scaffold types but was more abundant in the periphery than in the center of the scaffolds and for HFIP scaffolds mainly restricted to single pores.