This content is now available from Statistical Associates Publishers. Click here.
Below is the overview and table of contents in unformatted form.
Overview Researchers must demonstrate instruments are reliable since without reliability, research results using the instrument are not replicable, and replicability is fundamental to the scientific method. Reliability is the correlation of an item, scale, or instrument with a hypothetical one which truly measures what it is supposed to. Since the true instrument is not available, reliability is estimated in one of four ways: " Internal consistency: Estimation based on the correlation among the variables comprising the set (typically, Cronbach's alpha) " Split-half reliability: Estimation based on the correlation of two equivalent forms of the scale (typically, the Spearman-Brown coefficient) " Test-retest reliability: Estimation based on the correlation between two (or more) administrations of the same item, scale, or instrument for different times, locations, or populations, when the two administrations do not differ on other relevant variables (typically, the Spearman Brown coefficient) " Inter-rater reliability: Estimation based on the correlation of scores between/among two or more raters who rate the same item, scale, or instrument (typically, intraclass correlation, of which there are six types discussed below). These four reliability estimation methods are not necessarily mutually exclusive, nor need they lead to the same results. All reliability coefficients are forms of correlation coefficients, but there are multiple types discussed below, representing different meanings of reliability and more than one might be used in single research setting. Overview 7 Key Concepts and Terms 8 Scores 8 Number of scale items 8 Models 8 SPSS 8 SAS 9 Triangulation 9 Calibration 9 Internal consistency reliability 10 Cronbach's alpha 10 Overview 10 Interpretation 10 Cut-off criteria 10 Formula 10 Number of items 11 Cronbach's alpha in SPSS 11 Example 1 11 Alpha if deleted 11 Item-total correlation 12 R-squared 12 Negative alphas 12 KR2 12 Example 2 13 Standardized item alpha 13 Cronbach's alpha in SAS 14 SAS syntax 14 SAS output 14 Other internal consistency reliability measures 15 Ordinal reliability alpha 15 Raykov's reliability rho (?) 16 Armor's reliability theta 16 Spearman's reliability rho 16 Split-half reliability 17 Split-half reliability in SPSS 17 Overview 17 SPSS menu selections 17 Spearman-Brown split-half reliability coefficient 18 Guttman split-half reliability coefficient 20 Guttman's lower bounds (lambda 1-6) 20 Split-half reliability in SAS 21 Odd-Even Reliability 21 Overview 21 SPSS 22 Test-retest reliability 23 Overview 23 Inter-rater reliability 24 Overview 24 Cohen's kappa 24 Kappa in SPSS 24 Example 25 Interpretation 25 Weighted Kappa 26 Intraclass correlation (ICC) 26 Example 26 Sample size: ICC vs. Pearson r 27 Data setup 27 Interpretation 28 Obtaining ICC in SPSS 29 Single versus average measures 29 ICC Models 30 ICC use in other contexts 32 Assumptions 32 Additivity 32 Independence 33 Uncorrelated error 33 Consistent coding 34 Random assignment of subjects 34 Equivalency of forms 34 Equal variances 34 Similar difficulty of items 35 Same assumptions as for correlation 35 Frequently Asked Questions 35 How is reliability related to validity? 35 How is reliability related to attenuation in correlation? 35 How should a negative reliability coefficient be interpreted? 36 What is Cochran's Q test of equality of proportions for dichotomous items? 37 What is the derivation of intraclass correlation coefficients? 37 What are Method 1 and Method 2 in the SPSS RELIABILITY module? 38 Bibliography 39