The habenula (Hb) is an epithalamic structure differentiated into two nuclear complexes, medial (MHb) and lateral habenula (LHb). After decades of relative neglect, interest in the Hb resurged when it was demonstrated that LHb neurons play a key role in encoding disappointments and expectation of punishments. Consistent with such a role, the LHb has been implicated in a broad array of functions and pathologic conditions, notably in mechanisms of stress and pain, as well as in the pathophysiology of mood disorders. So far, the vast majority of research involving the LHb has focused on its role in regulating midbrain dopamine release. However, the LHb is also robustly interconnected in a reciprocal manner with a set of rostral serotonin (5-HT) nuclei. Thus, there is increasing evidence that the LHb is amply linked to the dorsal (DR) and median raphe nucleus (MnR) by a complex network of parallel topographically organized direct and indirect pathways. Here, we summarize research about the interconnections of the LHb with different subregions of the DR and MnR, as well as findings about 5-HT-dependent modulation of LHb neurons. Finally, we discuss the contribution of distinct LHb-raphe loops to stress and stress-related psychiatric disorders including anxiety and depression.