Several studies have demonstrated substantial variability among individual radiation oncologists in defining target volumes using computed tomography (CT). The objective of this study was to determine the impact of combined positron emission tomography and computed tomography (PET/CT) on inter-observer variability of target volume delineation in rectal cancer. We also compared the relative concordance of two PET imaging tracers, 18F-fluorodeoxyglucose (FDG) and 18F-fluorodeoxythymidine (FLT), against conventional computed tomography (CT). Six consecutive patients with locally advanced rectal cancer were enrolled onto an institutional protocol involving preoperative chemoradiotherapy and correlative studies including FDG- and FLT-PET scans acquired in the treatment position. Using these image data sets, four radiation oncologists independently delineated primary and nodal gross tumor volumes (GTVp and GTVn) for a hypothetical boost treatment. Contours were first defined based on CT alone with observers blinded to the PET images, then based on combined PET/CT. An inter-observer similarity index (SI), ranging from a value of 0 for complete disagreement to 1 for complete agreement of contoured voxels, was calculated for each set of volumes. For primary gross tumor volume (GTVp), the difference in estimated SI between CT and FDG was modest (CT SI = 0.77 vs. FDG SI = 0.81), but statistically significant (p = 0.013). The SI difference between CT and FLT for GTVp was also slight (FLT SI = 0.80) and marginally non-significant (p < 0.082). For nodal gross tumor volume, (GTVn), SI was significantly lower for CT based volumes with an estimated SI of 0.22 compared to an estimated SI of 0.70 for FDG-PET/CT (p < 0.0001) and an estimated SI of 0.70 for FLT-PET/CT (p < 0.0001). Boost target volumes in rectal cancer based on combined PET/CT results in lower inter-observer variability compared with CT alone, particularly for nodal disease. The use of FDG and FLT did not appear to be different from this perspective.