High sulfate concentrations in the groundwater occur in several cities and particularly in Berlin, Germany. Building rubble–composed soils and landfills are a major source of dissolved sulfates. This study assesses the sulfate release dynamics of such rubble-composed substrates. The substrate was taken from a building rubble landfill in Berlin, which was created after World War II. It was poured into two lysimeters, which were irrigated repeatedly for 2 years to simulate several years of groundwater recharge. Sulfate concentration in the leachate was measured monthly. Sulfate release dynamics were effectively described with PHREEQC, assuming either one or two sulfate pools with kinetically limited dissolution. The volume that percolated the lysimeter column was 2,440 L, corresponding to 17 years of local groundwater recharge. At the beginning, sulfate concentrations increased from approximately 10·10−3 mol ·L−1 to 13·10−3 mol · L−1, which is close to gypsum solubility concentration. After 1.8 pore volumes, a decrease was observed, and after 8 pore volumes, concentrations were relatively constant at levels less than 3·10−3 mol · L−1. The data were best described by a model that included a kinetically limited dissolution of gypsum from two sulfate pools different in their effective surface areas. One pool can be ascribed to fine-grained gypsum particles, whereas the other can be ascribed to coarse-grained ones. Overall, rubble-composed substrates can be a severe long-term source of sulfates.