Idiopathic pulmonary fibrosis (IPF) is the most common of the idiopathic interstitial pneumonias. It is typically associated with extensive and progressive fibrosis, and is fatal and has limited treatment options. Characteristically IPF patients display large lymphocyte aggregates composed of CD3+T cells and CD20+B cells within the lung tissue that are located near sites of active fibrosis. In addition, IPF patients can have autoantibodies to a range of host antigens, suggesting a breakdown in immunological tolerance. In this review, we examine the role of T and B cells in IPF pathogenesis and discuss how loss of self-tolerance to lung-specific proteins could exacerbate disease progression in IPF. We discuss what these results mean in terms of future prospects for immunotherapy of IPF.