Interobserver agreement, reliability, and generalizability of data collected in observational studies

    loading  Checking for direct PDF access through Ovid


Research in developmental and educational psychology has come to rely less on conventional psychometric tests and more on records of behavior made by human observers in natural and quasi-natural settings. Three coefficients that purport to reflect the quality of data collected in these observational studies are discussed: the interobserver agreement percentage, the reliability coefficient, and the generalizability coefficient. Three-facet generalizability studies that parallel intraobserver-interobserver, split-half, and test-retest reliability studies are described as examples. It is concluded that although high interobserver agreement is desirable in observational studies, high agreement alone is not sufficient to insure the quality of the data that are collected. Evidence of the reliability or generalizability of the data should also be reported. Other uses for generalizability theory (e.g., attribution of variance, single-S studies) are suggested, and further advantages of generalizability designs are discussed. (27 ref) (PsycINFO Database Record (c) 2006 APA, all rights reserved)

Related Topics

    loading  Loading Related Articles