Cryopyrin-associated periodic syndrome (CAPS) is a rare, heterogeneous disease entity associated with NLRP3 gene mutations and increased interleukin-1 (IL-1) secretion. Early diagnosis and rapid initiation of IL-1 inhibition prevent organ damage. The aim of the study was to develop and validate diagnostic criteria for CAPS. An innovative process was followed including interdisciplinary team building, item generation: review of CAPS registries, systematic literature review, expert surveys, consensus conferences for item refinement, item reduction and weighting using 1000Minds decision software. Resulting CAPS criteria were tested in large cohorts of CAPS cases and controls using correspondence analysis. Diagnostic models were explored using sensitivity analyses. The international team included 16 experts. Systematic literature and registry review identified 33 CAPS-typical items; the consensus conferences reduced these to 14. 1000Minds exercises ranked variables based on importance for the diagnosis. Correspondence analysis determined variables consistently associated with the diagnosis of CAPS using 284 cases and 837 controls. Seven variables were significantly associated with CAPS (p<0.001). The best diagnosis model included: Raised inflammatory markers (C-reactive protein/serum amyloid A) plus ≥two of six CAPS-typical symptoms: urticaria-like rash, cold-triggered episodes, sensorineural hearing loss, musculoskeletal symptoms, chronic aseptic meningitis and skeletal abnormalities. Sensitivity was 81%, specificity 94%. It performed well for all CAPS subtypes and regardless of NLRP3 mutation. The novel approach integrated traditional methods of evidence synthesis with expert consensus, web-based decision tools and innovative statistical methods and may serve as model for other rare diseases. These criteria will enable a rapid diagnosis for children and adults with CAPS.