Background: Both genetic and environmental factors have been reasoned for cancer development in schizophrenia patients. However, the influence of age of onset and duration of schizophrenia on cancer incidence has rarely been emphasized. Besides, bipolar disorder tends to resemble schizophrenia from the perspective of multiple rare mutations. Comparing pattern and risk of cancers between schizophrenia and bipolar patients is illuminating. Methods: This study used the Taiwan National Health Insurance Database. A total of 71 317 schizophrenia and 20 567 bipolar disorder patients from 1997 to 2009 were enrolled. Both cohorts were followed up for cancer during the same period by record linkage with the cancer certification in Taiwan. Age and gender standardized incidence ratios (SIRs) of overall and site-specific cancers were calculated. Results: The SIR for all cancers was 1.17 for the schizophrenia cohort. Increased cancer risk (SIR: 1.31, 95% CI: 1.17–1.48) was observed in females but not males. For the bipolar disorder cohort, the SIR for all cancers was 1.29, but the excess risk was found in males (SIR: 1.42, 95% CI: 1.14–1.77) and not females. Cancer risk decreases as the duration and age of onset of schizophrenia increases. If schizophrenia is diagnosed before 50, the SIRs for colorectal, breast, cervical, and uterine cancers increase but if diagnosed after 50, the SIRs for all cancers decrease except for breast cancer. In bipolar disorder, the SIRs for all site-specific cancers were insignificant. Conclusions: Among schizophrenia patients, overall cancer risk varies inversely with age at diagnosis and disease duration. Besides, gender-specific cancer risks differ between schizophrenia and bipolar disorder.