Data Imputation for Missing Values

Overview Proper handling of missing values is important in all analyses and is critical in some, such as time series analysis. Improper handling of missing values will distort analysis because, until proven otherwise, the researcher must assume that missing cases differ in analytically important ways from cases where values are present. That is, the problem with missing values is not so much reduced sample size as it is the possibility that the remaining data set is biased. The imputation of values where data are missing is an area of statistics which has developed much since the 1980s. Several imputation algorithms are now supported by SPSS's Missing Values option. Note that maximum likelihood data imputation can also be implemented in AMOS, the structural equation program supported by SPSS.

Contents Key concepts and terms Estimation methods Frequently asked questions Bibliography

Key Concepts and Terms

• Item non-response. Item non-response refers to the fact that due to fatigue, sensitivity, lack of knowledge, or other factors, respondents not infrequently leave particular items blank on mail questionnaires or decline to give any response during interviews. This forces the researcher to decide whether to leave cases with missing data out of analysis when data are missing for a variable being analyzed, or if a value should be imputed for the case and the blank replaced by the imputed value. Similar issues arise with archival data, where the researcher may find no recorded data for certain values of certain records.
  There is no simple decision rule for whether to leave data as they are, to drop cases with missing values, or to impute values to replace missing values. When the number of cases with missing data is small (ex., <5% in larger samples), it is common simply to drop these cases from analysis. This is done by default by SPSS, which uses listwise deletion (discussed below) Even when there are larger numbers of cases with missing data, the researcher may opt for dropping cases rather than imputing values when the purpose is multivariate analysis, as imputation can distort coefficients of association and correlation relating variables (see Kalton and Kasprzyk, 1982). An alternative is to run the analysis with dropped cases and with imputed values, to determine what difference the two methods may make, and this is recommended when imputed values are used for multivariate analysis.

  For purposes of univariate analysis (ex., understanding the frequency distribution of how subjects respond to an opinion item) imputation can reduce bias and often is used for this purpose if data are missing at random. There are two forms of randomly missing data, MCAR and MAR:

• Types of Missingness
  • Missing completely at random (MCAR) exists when missing values are randomly distributed across all observations. MCAR can be confirmed by dividing respondents into those with and without missing data, then using t-tests of mean differences on income, age, gender, and other key variables to establish that the two groups do not differ significantly. The SPSS Missing Values Analysis (MVA) option support's Little's MCAR test, which is a chi-square test for missing completely at random. If the p value for Little's MCAR test is not significant, then the data may be assumed to be MCAR. In SPSS, select Analyze, Missing Value Analysis, and check EM estimation. Little's test will be printed below the EM Means, EM Covariances, and EM Correlations tables (and will have the same value in each).

    If data are MCAR, then the researcher may choose listwise or pairwise deletion of cases. If data are not MCAR, missing values should be imputed. See "Estimation methods" below.

  • Missing at random (MAR) is a condition which exists when missing values are not randomly distributed across all observations but are randomly distributed within one or more subsamples (ex., missing more among whites than non-whites, but random within each subsample). We would like the key independent variables to be MAR with respect to the dependent variable. MAR is much more common than MCAR.
    • Testing for missing at random. The SPSS missing values analysis (MVA) module is accessed in SPSS by selecting Analyze, Missing Value Analysis. By default it will generate a table of "Separate Variance t Tests"in which rows are all variables which have 1% missing or more, and columns are all variables. In any cell, if P(2-tail) <=.05 this means that missing cases in the row variable are significantly correlated with the column variable and thus are not missing at random. If cases were missing at random they might be dropped, but otherwise imputation of values is the usual course of action.

      It should be noted that imputation has not been common in social science since statistical objections can be raised about any of the methods which might be used. An example of imputation for MAR data, used by the Bureau of the Census for instance, is the hot deck method, which simply means that the value of the case with the most recent non-missing value from the same subsample as the current case is used to replace the missing value in the current case. Stata and some other statistical packages implement the hotdeck method, under which the user specifies which variables form the strata (these variables should correlate with the item for which values are to be estimated), then multiple samples are taken from each stratum to derive the estimate of the missing value for the given item for cases in that stratum. Maximum likelihood estimation (MLE) is also used for MAR imputation. However, since the 1980s maximum likelihood and multiple imputation methods of imputation have been developed which may change this generalization.

  • Non-ignorable missingness is the most problematic form, existing when missing values are not randomly distributed across observations, but the probability of missingness cannot be predicted from the variables in the model. One approach to non-ignorable missingness is to impute values based on data otherwise external to the research design, as, for instance, estimating race based on Census block data associated with the address of the respondent.

  
  
• Estimation methods The SPSS MVA module(under Analyze, Missing Value Analysis) allows the user to select four estimation methods: listwise, pairwise, regression, and EM (maximum likelihood estimation, see below), and there are others. SPSS , under Transform, Replace Missing Cases, supports five estimation methods: replacement of missing values with the series mean, by the mean or median of nearby points, or linear interpolation between prior and subsequent known points, interpolating between the adjacent valid values above and below the missing one, or substitution of the linear regression trend value for that point (missing values are replaced with their predicted values).
  • Pattern analysis.The SPSS MVA module has a "Patterns" button, which allows the user to specify any of three types of pattern tables, in which X's indicate missing values, S's indicate system-missing values, pluses indicate extremely high values, minuses indicate extremely low values, and letters A - C can indicate user-specified missing values. The Patterns choice tabulates the frequency of each type of missing value pattern. For cases with missing values, case by variable patterns can be displayed. Cases and variables can be sorted by similarity of patterns. For tabulated patterns, one can list the mean of quantitative variables or, for categorical variables, the number of cases having the pattern in each category.

    The MVAR module in SPSS displays three types of 'indicator variable statistics' under its 'Descriptives' options, where the indicator variable is a dichotomous variable SPSS creates to flag whether or not the value of a given variable is present or missing.

    1. Crosstabulations: For each categorical variable, a table is generated showing for each category of the categorical variable the frequency and percent of nonmissing values for other variables.
    2. Percent mismatch: For each pair of variables, the percentage of cases in which one variable has a missing value and the other variable has a nonmissing value is displayed. This means also that the diagonal element in the table contains the percent of missing values for a single given variable.
    3. t tests : The means of the two groups formed by indicator variables are compared for each quantitative variable. The t statistic, degrees of freedom, counts of missing and nonmissing values, and means of the two groups are displayed. By default, indicator variables with fewer than 5% missing are not displayed, but the user may set this percentage under Descriptives. The probability of t tells the researcher whether the means of a given variable are different for the two groups formed by present vs missing values on a second variable. SPSS computers t and the corresponding df, but does not compute the p value.

  • Listwise vs. pairwise deletion. Computer packages offer the choice of listwise vs. pairwise deletion of cases with missing values. Sometimes listwise is labeled casewise. Listwise deletion omits cases which do not have data on all variables in the variables list of the current analysis (ex., the current SPSS run) . Listwise deletion is preferred over pairwise deletion when sample size is large in relation to the number of cases which have missing data. However, even listwise deletion is considered an inefficient method which leads to bias. Both listwise and pairwise methods assume missing values are MCAR.

    Pairwise deletion omits cases which do not have data on a variable used in the current calculation only. This means that different calculations (ex., different correlation coefficients) will utilize different cases and will have different sample sizes (different n's). This effect is undesirable (and in some procedures like structural equation modeling, may prevent a solution altogether), but pairwise deletion may be necessary when overall sample size is small or the number of cases with missing data is large. Even then, misinterpretation may well result unless missing data are missing completely at random (MCAR).

    Selecting "listwise" in SPSS displays the means, correlation matrix, and covariance matrix, omitting cases that have missing values in any variable under consideration (listwise deletion). Selecting "pairwise" in SPSS displays for each pair of quantitative variables the number of pairwise nonmissing values, and the pairwise mean, variance, covariance, and correlation.

  • Mean substititution was once the most common method of imputation of missing values but is no longer preferred. Substitution of the simple (grand) mean will reduce the variance of the variable. Reduced variance can bias correlation downward (attenuation) or, if the same cases are missing for two variables and means are substituted, correlation can be inflated. That is, this method creates a spiked distribution at the mean in frequency distributions and causes attenuation in correlation of the item with others, and underestimates variance. These effects on correlation carry over in a regression context to lack of reliability of the beta weights and of the related estimates of the relative importance of independent variables. That is, mean substitution in the case of one variable can lead to bias in estimates of the effects of other or all variables in the regression analysis, because bias in one correlation can affect the beta weights of all variables. Somewhat better is substitution of the group mean for a categorical (grouping) variable known to correlate highly with the variable which has missing values. By and large, however, mean substitution is no longer recommended.

  • Multiple regression may be used for data imputation simply by using non-missing data to predict the values of missing data. Note that this may "over-correct," introducing unrealistically low levels of noise in the data. The regression method has the problem that all cases with the same values on the independent variables will be imputed with the same value on the missing variable, causing a portion of the same problems as mean substitution. Consequently, the preferred method is stochastic substitution, which uses the regression technique but adds a random value to the predicted result. This random value is usually the regression residual from a randomly selected case from the set of cases with no missing values. By default, in regression estimates, SPSS adds the residual of a randomly picked case to each estimate, though the user can select residuals, normal variates, Student's t variates, or no adjustment. Regardless, this residual is just a guess and it is likely that standard errors (and hence confidence intervals and probability values) will be smaller than they should be had no imputation been necessary. The regression method assumes missing values are MAR (as opposed to MCAR). The regression method also assumes that the same model explains the data for the non-missing cases as for the missing cases, which, of course, is not necessarily true. Finally, the user can also set a maximum limit on the number of predictor variables used for the estimate, as the larger the number of predictors, the greater the chance that the imputed estimate is modeling noise in the data rather than an actual relation of model variables to missingness.

  • Maximum likelihood estimation (MLE), such as implemented by the EM algorithm in the SPSS Missing Values option, applies MLE to the task of imputing missing data values without recourse to the simulation involved in multiple imputation methods discussed below. MLE makes fewer demands of the data in terms of statistical assumptions and is generally considered superior to imputation by multiple regression. This is now the most common method of imputation. The MLE method assumes missing values are MAR (as opposed to MCAR) and shares with multiple regression the problem of over-correction and possible modeling of noise.

    In SPSS the user can specify the data distribution assumptions to be used by the EM algorithm: normal, mixed normal, and Student's t. For a mixed normal assumption, the user can specify the proportion and the standard deviation ratio. The user can also set the maximum number of iterations attempted in the iterative MLE process used by the EM algorithm (not recommended as the procedure may then stop even if estimates have not yet converged).

  • Approximate Bayesian bootstrap (ABB) methods use logistic regression to estimate the probability of response/non-response on the dependent variable y, based on covariates X_i. One can then group cases into quintiles or similar categories based on this estimate of probabilities of response/non-response. Within each category one can then create a new pool of respondents by sampling values from the non-missing cases with replacement, and can create a set of imputed data by sampling this new pool with replacement as many times as there are missing cases in that quintile. Doing this for each quintile allows the researcher to add imputed data to the observed data prior to conducting analysis. SOLAS software implements the ABB method (Statistical Solutions Ltd., County Cork, Ireland). However, note that ABB is appropriate when the researcher's purpose is analyzing Y but not when the purpose is assessing interrelationships among the independent X's, as is often the case in regression or SEM modeling. SPSS does not yet support approximate Bayesian bootstrap methods.

  • Multiple imputation (MI) is a method of generating multiple simulated values for each incomplete datum, then iteratively analyzing datasets with each simulated value substituted in turn. The purpose is, arguably, to generate estimates that better reflect true variability and uncertainty in the data than do regression methods. Multiple imputation methods yield multiple imputed replicate datasets (5 is typical) each of which is analyzed in turn. The results are combined and the average is reported as the estimate. The standard error of estimate is equal to SQRT[(1 - 1/m)/B +W], where m is the number of replicates, B is the variance of the imputations, and W is the average of the estimated variances (Pickles, 2005: 693). Simulated values are generated by Monte Carlo methods. MI has the advantage of simplicity over MLE, making it particularly suitable for large datasets. In this light it has been shown that the efficiency of data imputation using MI is high even when the number of imputed datasets is low (in the 3 to 10 range). Also, tests suggest that MI is quite robust even when the simulation is based on an erroneous model (ex., when normality is assumed for purposes of simulation when the underlying data are not in fact normal). SPSS does not yet support multiple imputation.

Frequently Asked Questions

• Why not just delete cases with missing values rather than impute values at all?
  Little and Rubin (1987), among others, have demonstrated the dangers of simply deleting cases. Case deletion strategies assume that the deleted cases are a relatively small proportion of the entire dataset and are representative of it -- that is, when cases are missing completely at random. In most research settings, however, missing data are indicative of some pattern and cannot safely be assumed to reflect randomness. In such circumstances, deletion can introduce substantial bias into the study. Moreover, the loss in sample size can appreciably diminish the statistical power of the analysis. As a rule of thumb, if a variable has more than 5% missing values, cases are not deleted, and many researchers are much more stringent than this.

• Should I use original data or imputed data when reporting results?
  The original dataset may be biased by a large number of non-random missing values. The imputed dataset is a "what-if" hypothetical dataset which relies on estimation, which is imperfect and itself may present biases, though it is a "best guess" attempt to present what choices respondents are likely to have made, given their responses on other items. Some authors base statistical reporting on the original dataset, noting in the methods section the possible biases based on an analysis of missing responses. Other authors present reports based on imputed data, giving the rationale for imputation and stating possible biases in the methods section. This author recommends reporting results based on the original dataset, but running all analyses on both the original and imputed datasets, and discussing in the report where imputation would make a difference for the substantive (not merely statistical) interpretations the author wishes to make.

• Where can I get software for data imputation, apart from SPSS?
  Joe Schafer, a leading authority in the field, has made available several free MI software programs for data imputation, both for the statistical package S-Plus and as stand-along programs. The NORM program is the central one for MI.

  Solas (Statistical Solutions, 1998) is imputation software based on the ABB algorithm describred above, distributed by Statistical Solutions Ltd.

Bibliography

• Allison, P. D. (2001). Missing data. Thousand Oaks, CA: Sage Publications.
• Little, R. J. A. and D. B. Rubin (1987). Statistical analysis with missing data. John Wiley & Sons, New York.
• Pickles, Andrew (2005). Missing data, problems and solutions. Pp. 689-694 in Kimberly Kempf-Leonard, ed., Encyclopedia of social measurement. Amsterdam: Elsevier.
• Rubin, D.B. (1987). Multiple imputation for nonresponse in surveys. John Wiley & Sons, New York.
• Rubin, D.B. (1996). Multiple imputation after 18+ years. Journal of the American Statistical Association 91: 473-489.
• Schafer, J.L. (1997) Analysis of incomplete multivariate data. Chapman & Hall, London. Book No. 72, Chapman & Hall series Monographs on Statistics and Applied Probability.
• Schafer, J.L. (1999). Multiple imputation: A primer. Statistical Methods in Medical Research. 8: 3-15.
• Schafer, J.L. and M. K. Olsen (1998). Multiple imputation for multivariate missing-data problems: A data analyst's perspective. Multivariate Behavioral Research. 33: 545-571.