Aging effects on regional brain activation have been studied extensively to explain the gradual recollection failure that occurs with advancing age. However, little is known about the consequence of aging on the interaction among brain regions that support recollection. The purpose of this study was to examine effective connectivity at encoding and retrieval during successful and unsuccessful recollection in young and old adults. In particular, we analyzed a recollection network that is characterized by its susceptibility to aging effects by middle age or later, which is comprised of the occipital cortex, hippocampus and orbitofrontal cortex. Participants' brains were scanned using functional magnetic resonance imaging while they performed a spatial source memory task. Dynamic causal modeling and Bayesian model selection revealed that subsequent recollection during encoding and recollection during retrieval modulated the influence of the orbitofrontal cortex on the hippocampus in both age groups; this particular connectivity was not modulated by unsuccessful encoding in either group. Successful encoding and retrieval of item-source associations modulated all connections within the network in old adults. The findings revealed that the orbitofrontal cortex influences processes in the hippocampus to ensure successful recollection, and aging alters the recollection network by engaging non-specialized connections.