Latent Class Analysis

Overview Latent class analysis (LCA) is a multivariate technique which can be applied for cluster, factor, or regression analytic purposes. Latent constructs are created from indicator variables, as in structural equation modeling, and then used to construct clusters, factors, or to predict dependents in regression mode. LCA is commonly used to classify cases into a set of latent classes, based on categorial indicator variables such as survey items. After cases are so classified, then in regression mode, LCA with covariates uses additional variables to predict class membership. Latent profile analysis is a variant on LCA for continuous variables. Log-linear modeling with latent variables is a form of analysis combining latent class analysis with log-linear analysis. Latent class analysis software such as Latent Gold implement latent class models for cluster analysis, factor analysis, and regression with latent classes. Unlike traditional models, such latent class models support nominal and ordinal as well as continuous data. Mixture modeling with structural equation models is a major type of latent class analysis and is discussed separately.

Contents Key concepts and terms Assumptions Model fit Example output Frequently asked questions Bibliography

Key Concepts and Terms

• Latent classes are the dimensions which structure the cases with respect to a set of variables. When all latent classes are controlled, only a random relationship among variables remains. That is, latent class analysis divides the cases into latent classes which are "conditionally independent," meaning that the variables of interest are uncorrelated within any one class. Variations on LCA exist to handle the situation where exceptions to conditional independence exist (ex., all but two variables of interest are uncorrelated within any one class). Classes are "latent" because they are not observable directly but rather are identified based on a function of a set of manifest indicators.

• Response vectors are simply the sets of responses of a subject to a set of items, ex.: {0111] for four binary items coded 0 or 1.

• Model parameters in an LCA model are maximum-likelihood estimates (MLE) of conditional response probabilities (probabilities for each latent class that a member of that class will display a given value on a manifest variable). The estimated proportion of cases in each latent class, summing to 1.0, is also a model parameter in LCA. MLE estimation iteratively calculates model parameters to be those which are most likely to account for observed results. Note MLE estimation precedes the actual allocation of cases to latent classes. See Dayton (1998): 12-14 on MLE.

• Number of latent classes. The number of latent classes may be determined in two ways. The first and most common method is to iteratively test the goodness of fit of models with 2, 3, ...,n latent classes using the likelihood ratio chi-square test discussed below. (In SAS PROC LCA, for instance, the research can specify 2, 3, ...n classes to model). As classes are added, model fit will tend to improve, and one may set the number of latent classes as the minimum number possible while still obtaining a finding of significance of model fit. A second method is bootstrapping. In SAS PROC LCA, the research can specify the number of classes to model.

• Classification of cases. The posterior probability of membership in latent class t, given response vector y for subject s equals a ratio in which the numerator is the product of the latent class proportion times the probability of response vector y_s assuming membership in latent class t; and the denominator is the unconditional probability for response vector y. See Dayton (1998): 9 ff. for formulas. Computationally, Bayes' theorem is applied for each latent class and then all cases with a given response vector are classified into the latent class for which the posterior probability is highest. Put another way, "recruitment probabilities" are used to actually classify cases into the latent classes. The probabilities are a function of the model's parameters (estimated conditional response probabilities and estimated prevelance of each latent class). Each case is assigned to the latent class for which it has the highest a posteriori (Bayesian) probability of membership. Alternatively, the same methods can assign each case a probability of membership in each latent class.

• Gamma estimates are the class membership probabilities reported for each categorical independent variable. If there are three classes, then PROC LCA output will report three gamma estimates, one for each class, interpreted as estimates of the the percent of observations in each class. The gamma estimates will add to 1.0 (barring rounding errors).
  • Using gamma estimates to assess categorical covariates. The researcher can divide the sample into the levels of any given categorical covariate (ex., gender = 0, 1) and then obtain gamma estimates for each subsample. This enables the researcher to see the relation of levels of the covariate with classes of the response variable). For instance, the gamma estimate for gender=0 may be high for class 1 of the dependent and low for other classes, while the gamma estimate for gender=1 may show a different pattern.

  • Gamma estimates vs. odds ratios. While both odds ratios (discussed below) and gamma estimates may be used to assess a categorical covariate, they differ in meaning. Odds ratios are interpreted relative to the left-out reference class of the response variable, whereas gamma estimates are an absolute (non-relative) measure of class membership probability. Differences in gamma estimates among different levels of a categorical covariate reflect absolute effects. Differences in odds ratios among different levels of a categorical covariate reflect effects relative to a reference category.

  • Rho estimates are the item response probabilities. As illustrated in the output example below, the rho estimates provide information about the characteristics of the dependent variable as measured by the response indicators, for each of the requested number of classes. When covariates are added to a model, the item response probabilities (rho estimates) should remain relatively stable to indicate a well-fitting model (Lanza et al., 2007).

  • Odds ratios measure the effect sizes of the covariates in the model. They are calculated by running multinomial regression with the latent class variable as dependent , using each covariate in turn as independent. There will be one odds ratio per class except for the last, which is the reference class. The odds ratio is "the odds of being in a given class relative to the reference class for a given level of the covariate, divided by the odds of being in the given class relative to the reference class for one unit less than the given level of a covariate" (Bray, 2008).

    An odds ratio of, say, 1.5 for class 1 means that a one unit increase in the covariate corresponds to a 50% greater likelihood (that is, multiplies the odds by a factor of 1.5) of a case being in class 1 compared to being in the reference class. Odds ratios, as in other applications, are thus effect size measures where 1.0 indicates no effect (independence) and the greater above 1.0, the greater the positive effect size associated with that covariate. Below 1.0 reflects negative effect.

  • Posterior probabilities are the probabilities of an event given certain prior probabilities.The posterior probability for a given observation indicates the probability of that observation being classified in a given class. For example, if there are three classes, each case will have three posterior probabiities, indicating the probability of classifying the given case in class 1, 2, or 3. Typically in survey research, the probabilites are based on respondents' responses to a set of survey items. These are used to assign individuals to classes. If class membership assignments are reliable, then the relation of covariates (additional variables) to class membership may be assessed.
    • Saving posterior probabilities. In SAS PROC LCA, the researcher must specify in advance the number of classes for the solution. For each class, a posterior probability can be computed. For three classes, PROC LCA will create these under the labels POSTLC1, POSTLC2, and POSTLC3. These posterior probabilities can be saved to file and manipulated by spreadsheets or other programs.

    • Assessing the reliability of class assignments with mean posterior probabilities. Posterior probabilities can be used to assess the reliability of LCA class assignments. Using a spreadsheet, the researcher can sort on class membership and the compute the mean posterior probability of POSTLC1, POSTLC2, and POSTLC3 for those classified in any given class. In the ideal and totally unambiguous classification situation, those in class 1 will have a mean posterior probability of 1.0 on POSTLC1, and a mean posterior probability of 0.0 on POSTLC2 and POSTLC3. The closer these means are to 1.0 for the class 1 classified cases on POSTLC1, for class 2 cases on POSTLC2, and for class 3 cases on POSTLC3, the more unambiguous the class assignments. Daniel Nagin suggests that the average assignment probability for each class should be over 0.90 for the assignment to be acceptable (Bray, 2008). By extension, if class assignments are not acceptable, then subsequent statistical analysis using class membership as a variable will also be unacceptable; instead the model should be revised to achieve acceptable levels of class assignment.

    • Assessing the relation of covariates to class membership. Using a spreadsheet, the researcher can sort on a covariate, then get the mean posterior probabilities (ex, of POSTLC1, POSTLC2, and POSTLC3 for a three-class solution) for cases where covariate=1; then for cases where covariate=2; etc. The means will approximate the proportions within the covariate category assigned to each class (for ex., the percentages in each of three classes for covariate=1. The more these means are the same, the more the covariate is independent of class membership. Note that while odds ratios may be used to assess independence of a categorical covariate, gamma estimates discussed below are generally used to assess effect size of a categorical covariate..

    • Computing odds ratios from posterior probabilities. While in principle it is possible to compute odds ratios from the posterior probabilities, this method of estimation will be in error to the extent that the posterior means are less than 1.0 (meaning that the posterior probabilities are not all 1.0 for a given class and 0.0 for all other classes for any given case). Because measurement error is common in real-world data, this method of computing odds ratios may lead to severely misleading estimates and is not recommended.

• Latent class cluster analysis. In Latent Gold, the latent class cluster module in Latent Gold differs from traditional cluster analysis algorithms, which group cases near each other by some definition of distance. The latent class approach defines one cluster per latent class, using model-based probabilities to classify cases. Covariates can be used in the model. In Latent Gold, the Variables tab is used to specify the number of classes for cluster models. One may also specify a range to estimate several models (ex., enter "1-4" to estimate four latent class models containing 1, 2, 3, and 4 clusters respectively). Options exist in Latent Gold to constrain the model in various ways. For cluster models, the Clusters tab of the Analysis dialog box allows the user to change the number of clusters or to restrict various effects to equal zero (beta effects of the indicators on the latent variable(s); gamma effects of selected covariates on the latent variable). The Allow Model Effects box is used to specify a cluster model where beta and gamma effects on the latent variable are estimated only for selected variables, with the remaining effects being restricted to 0 simply by checking a checkbox.

• Latent class factor analysis. In Latent Gold, latent class factor analysis treats the latent classes as factors, but avoids such problems of traditional factor analysis as having to rotate factors and having to assume continuous observed variables. One may specify the number of factors for factor models, where each factor may have up to five levels. The Correlations box allows correlations between selected pairs of factors to be included in the factor model. Output may include biplots, which can locate values of nominal, ordinal, or continuous variables on factor axes.

  For factor models the Factors tab of the Analysis dialog box allows users to change the number of factors and/or factor levels, restrict factor effects to zero on selected variables, and to restrict selected factor correlations to zero. The Correlations box allows correlations between selected pairs of factors to be included in the factor model. The Technical tab of the Analysis dialog box contains options to control the number of iterations and convergence limits, start values, Bayes constants, treatment of missing data, and error variances. In cluster and factor models one may include direct effects in the model in the Output tab. A direct effect between two indicators and direct effects of selected covariates on selected indicators can be selected for inclusion in a model using the Residuals tab on the Analysis dialog box.

• Latent class regression analysis. Latent class regression analysis, discussed separately as a variant of mixture analysis in structural equation modeling, also known as latent class segmentation analysis. One set of items is used as in all LCA to establish class memberships, then additional covariates are used to model the variation in class membership. Latent class (mixture) regression uses predictor and covariate variables to estimate a statistical model which can assign cases to each latent class, all without the traditional regression assumptions of normally distributed prediction error or of homogeneity along the entire regression line. For regression models, specifying one class corresponds to traditional regression assuming a single homogeneous population. However, regression models may be estimated for 2, 3 and 4 classes (segments) respectively. If the dependent variable in a latent class regression model is set to ordinal, the adjacent category logit model, also known as the baseline category logit model, is specified. If the dependent variable is set to count, the Poisson model is used. If the dependent variable is set to Binomial Count, the binomial model is used. If the dependent is specified as continuous, an ordinary regression model is used.

  
  

• Measures of Model Fit
  • Model chi-square. Also called likelihood ratio chi-square, the model fit likelihood ratio chi-squared statistic, G², and L², model chi-square has a chi-square distribution with (S - p - 1) degrees of freedom, where S = the total number of different response patterns and p = the number of estimated model parameters. Model chi-square represents the amount of the relationship between the variables that remains unexplained by a model; the larger the value, the poorer the model fits the data. If the computed model chi-square is greater than the critical value of chisquare for the desired probability level (ex., .05), the the researcher fails to conclude that the model fits the data. As a rule of thumb, a good fit is provided by a model when model chi-square for a model is not substantially larger than the degrees of freedom.

  • Difference chi-square is simply the difference of two model chi-square's for two nested models, where degrees of freedom for difference chi-square is the difference in the two degrees of freedom for the models. A nested model is one with parameter restrictions compared to a full model (ex., constrained so that within each latent class, all the conditional probabilities should be equal). Because nesting is required, this is a form of "hierarchical analysis." If the computed difference chi-square is less than the critical value of chi-square for a given significance level (ex., .05), then the researcher concludes that one model has a significantly better fit to the data than the other model. See Dayton (1998): 17-18. Note that the chi-square difference cannot be used if the two models compared have different numbers of latent classes, even if one is nested within another. In Latent Gold, there is a ProbChi ("Chi-Square Calculator") tool which tests whether the simplification is warranted: if chi-square difference is not significant, the more parsimonious model (Model2) may be used without significant loss of information.

  • The Cressie-Read statistic is used to assess the validity of model-chi-square. If it does not yield a p values similar to model chi-square, the reported p-value for model chi-square may not be valid.

  • Goodness of fit measures. LCA employs many of the same model fit criterion as structural equation modeling, including BIC (Bayes information criterion), AIC (Aikike information criterion) and CAIC (Consistent AIC, which penalizes for sample size as well as model complexity). These are goodness of fit measures which take into account model parsimony (that is, it penalizes for number of parameters in relation to maximum possible number of parameters). The lower the BIC, AIC or CAIC values, the better the model in comparison with another. These goodness of fit measures are discussed in the section on SEM. See also, Dayton (1998): 18-20.
    • AWE. Similar to BIC, but also takes classification performance into account.

    • The classification error ratio, also called P_c, is the proportion correctly classified, equal to the posterior probability for the modal latent class for a given response vector. The closer to 0 the better.

    • Reduction Errors, Entropy R-squared and Standard R-squared (for factor models only). These indicate how well the model predicts class memberships or factor scores. The closer to 1 the better.

    • Classification log-likelihood: This is the log-likelihood value under the assumption that the true class membership is known.

    • I_D is the index of dissimilarity, whose lower limit is 0 and upper limit is < 1. A value of < .05 is considered a small difference.

• Parameters output: For any estimated model, click Parameters and Latent GOLD displays a table containing parameter estimates (betas, gammas, sigmas) and measures of significance for these estimates. Betas measure the effects of each indicator on the latent variable(s) (on each cluster or on each factor). For regression models, betas measure the effects of each predictor on the dependent variable in a way more comparable than unstandardized regression coefficients. Gammas reflect the effects of the selected covariates on the latent variable(s). That is, gammas are parameters of the logit model used to predict the latent distribution as a function of the covariates. Sigmas are error variances and covariances for continuous dependent variables and indicators.

• Standard errors are also displayed in Latent Gold for parameter estimates. Parameter estimates larger than 1.96 times their standard error are significant at the .05 level. Identifying non-significant parameters can be useful in model diagnostics and model modification.

• The Wald statistic is used to assess the statistical significance of the set of parameter estimates for a given variable. Specifically, it tests the hypothesis that each of the parameter estimates in that set equals zero. For regression models, by default, two Wald statistics (Wald, Wald(=)) where the Wald(=) statistic tests the equality of each set of regression effects across classes when more than 1 class has been estimated.

• Bi-plots (factor models only): For factor models with two or more factors, the factor means for any two selected factors may be plotted to form a Bi-Plot. Different symbols on the biplot show the location on the two selected factors of each value/category of indicators and covariates. By default, Factor1 is used as the horizontal axis and Factor2 as the vertical axis. Click on any variable symbol in the bi-plot and the status bar displays the corresponding variable name and category values. A biplot from the Latent Gold factor analysis module is shown below.
  

PROC LATCLASS example output

• PROC LATCLASS 3-class commented output

Assumptions

• Nonparametric. LCA does not assume linearity, normal distribution of data, or homogeneity of variances.

• Data level. Latent class analysis is appropriate when the dependent variable is truly categorical (ex., region of residence, candidate for whom voted in the last election). Often used with binary data (ex., has a type of cancer/doesn't, in medical research), it may also be used with ordinal data such as Likert scales, where there are a limited number of categories. A variation of LCA called latent profile analysis may be used with ordinal rank data and continuous variables. Models may combine categorical and continuous variables (this is supported by the MPlus statistical package, for instance).

• Identified models. If models are not identified, there will not be a single computational solution. This corresponds to the problem of having more unknowns than equations in a model. The maximum number of estimatable parameters in a model equals (p - 1), where p = the number of rating combinations (ex., three binary variables have 2*2*2 = 8 patterns, assuming a model with no constraints). The total unknowns equal the number of latent classes times the number of variables (these are the response probabilities) plus (c - 1) estimates of latent class proportions, where c is the number of latent classes in the model (1 is subtracted since the proportion of the last class is determined once all others are estimated). Thus an unconstrained model with three binary variables and two classes will be "just identified," with 7 estimatable parameters and 7 unknowns.However, the same model with three classes would not be identified, as it would have 7 estimatable parameters and 11 unknowns. Such a model may be made identified by adding equality or other constraints to the model. LCA computer programs may trap for and report model identification problems.

• A divisible population. It is assumed that the population may be divided into a finite number of classes which are mutually exclusive and exhaustive.

• Populated tables. The likelihood ratio chi-square cannot be used reliably to assess model goodness-of-fit if the table of response patterns is sparse. This typically happens when the researcher has many multi-category variables such that the number of possible rating combinations for the set of variables becomes much larger than the sample size.

• Nested models. The difference chi-square test of difference of fit for two models requires that one model be a subset of the other (that is, its model parameters are a subset of the other's), as when one one model is a constrained version of the other. This further implies that the two models must have the same number of latent classes.

• Conditional independence. Within each latent class, observations are assumed to be independent.

Frequently Asked Questions

• Why not just use factor analysis instead of LCA?
  While LCA may be thought of as a categorical analog to factor analysis, using ordinary factor analysis with categorical data is apt to lead to misleading parameter estimates and misleading goodness-of-fit indices (Vermunt & Magidson, 2005).

• What software implements LCA?
  Several packages implement LCA:
  • PROC LCA for SAS is a third-party add-on module for SAS, available for download from The Methodology Center, The Pennsylvania State University. To go to the download page, click here. See Lanza et al. (2007).
    
  • MLLSA is perhaps the most widely available. A microcomputer version is available from http://www.education.umd.edu/EDMS/Latent/Dayton.html.
    
  • LEM is a version which also handles loglinear analysis with latent variables, available from http://www.kub.nl/faculteiten/fsw/organisatie/departementen/mto/software2.html.
    
  • Latent Gold is a commercial package for latent class/mixture modeling. Contact Statistical Innovations, Inc., 375 Concord Ave., Belmont, MA 02478; 617-484-3700; http://www.statisticalinnovations.com; 617-484-3700. It supports latent class cluster, latent class factor, and latent class regression models, based on Vermunt and Magidson (2000). The latent class cluster module also supports traditional LCA.
    
  • MPlus also implements LCA: Mplus site.
    
  • John Ueberwsax's LCA website also lists software

• Does SPSS implement latent class analysis?
  Not as such (as of Version 10), but it offers mixture analysis in SEM and other procedures for factoring categorical data.

• What is multi-group LCA?
  This is an extension of LCA, supported by PROC LCA for SAS for instance, which allows computation of class membership separately for different subgroups, then to test for measurement invariance across groups to determine if class structure can be said to be the same, for example, for men and women.

• What is constrained LCA?
  This term refers to the fact that LCA software usually will allow the researcher to set certain requirements on the MLE solution for model parameters. For instance, the researcher may require that the conditional probabilities for two variables always be equal in any given latent class. Other probabilities may be set to a given fixed value, such as .50 for a given latent class.

• How is LCA related to assessing measurement error?
  LCA treats the items used to establish class membership as probabilistic, which is to say, as imperfect indicators of class membership. Some authors (ex., Flaherty, 2002) use LCA to gauge measurement errorin assigning cases to classes. See the discussion above regarding assessing the reliability of LCA class membership assignments. There is evidence that assessing measurement error in this manner is comparable to estimates obtained by reinterviewing to assess misreporting of class membership (see Biemer & Wiesen, 2002).

• Can latent class analysis handle hierarchical/multilevel data?
  Multilevel latent class analysis is available in Mplus but not yet in SPSS or SAS as of 2011.

• How does the Latent Gold package operate?
  The operation of Latent Gold is simple. The usual File, Open, command allows the user to load in SPSS .sav files directly. After loading data, one chooses cluster, factor, or regression from the ensuing dialog box. Selecting a model type opens the Analysis dialog box, which contains selections for variable selections, scale type, number of classes, and so on. Variable selections may be made for indicator, covariate, dependent, or predictor variables. Indicators define and measure the latent variables. Covariates control and describe latents. Dependents are observed variables which are predicted by latents. Predictors are additional observed variables used to predict the dependent(s). Once one has specified the model, in Latent Gold one clicks Estimate at the bottom of the Analysis dialog box or from the menus selects Model > Estimate. The program will estimate the model and create a new model name in the Outline pane. Additional models may be estimated for the same dataset by right clicking on the new model name or any previous model name in the Outline pane. After selection of variables, the Scan or Estimate commands cause Latent Gold to identify each distinct value for each selected variable. After scanning, double click on a variable to view frequency counts for variable values, and to assign category labels. The Scale Types command assigns scale types to the variables (Nominal, Ordinal, Continuous, Count, Binomial Count, Numeric). The Continuous scale type causes the normal distribution to be used. The Count scale type specifies the Poisson distribution. The default for indicators and dependent variables is ordinal when the variable contains numeric (quantitative) values, and nominal when the variable contains character codes (string variables). The default for covariates and predictors is numeric when the variable contains numeric (quantitative) values, and nominal when the variable contains character codes. The user can reassign scale types.

• Where can I find out more about LCA?
  • Keith Markus' LCA Website
  • The "Latent Class Analysis Website" by John Uebersax at http://john-uebersax.com/stat/index.htm.
  • See also the website of The Methodogy Center at Pennsylvania State University.

Bibliography

• Andersen, E. B. (1990). The statistical analysis of categorical data. Berlin: Springer.
• Biemer, P. P. & Wiesen, C. (2002). Measurement error evaluation of self-reported drug use: A latent class analysis of the U.S. National Household Survey on Drug Abuse. Journal of the Royal Statistical Society 165, 97-119.
• Bray, Bethany (2008). Email to author of 7/16/2008. (Bray is Assistant Director, The Methodology Center, The Pennsylvania State University, originators of PROC LCA).
• Chung, H., Flaherty, B. P., & Schafer, J. L. (2006). Latent class logistic regression: application to marijuana use and attitudes among high school seniors. Journal of the Royal Statistical Society, Series A, 169 (Part 4), 723-743. Illustrates computation of odds ratios.
• Clogg, C. C. (1995). Latent class models. In G. Arminger, C. C. Clogg, & M. E. Sobel (Eds.), Handbook of statistical modeling for the social and behavioral sciences (Ch. 6; pp. 311-359). New York: Plenum.
• Clogg, C. C. & Goodman, L. A. (1984). Latent structure analysis of a set of multidimensional contingency tables. Journal of the American Statistical Association 79, 762-771. With Goodman (1974), one of the seminal articles for LCA.
• Dayton, C. Mitchell (1998). Latent class scaling analysis. Quantitative Applications in the Social Sciences Series No. 126. Thousand Oaks, CA: Sage Publications.
• Flaherty, B, P. (2002). Assessing the reliability of categorical substance use measures with latent class analysis. Drug and Alcohol Dependence 68, S7-S20.
• Goodman, L. A. (1974), "Exploratory Latent Structure Analysis Using Both Identifiable and Unidentifiable Models," Biometrika, Vol. 61: 215-231.
• Hagenaars, J. A. (1993). Loglinear models with latent variables.Quantitative Applications in the Social Sciences series #94. Thousand Oaks, CA: Sage Publications.
• Lanza, Stephanie T.; Collins, Linda M.; Lemmon, David R. & Schafer, Joseph L. (2007). PROC LCA: A SAS procedure for latent class analysis. Structural Equation Modeling 14(4), 671 - 694.
• Lazarsfeld, P. F. & Henry, N. W. (1968), Latent Structure Analysis, Boston: Houghton Mifflin.
• McCutcheon, A. L. (1987). Latent class analysis. Quantitative Applications in the Social Sciences Series No. 64. Thousand Oaks, CA: Sage Publications.
• McCutcheon A, Hagenaars J., eds. (1999). Advances in Latent Class Modeling. Cambridge, UK and NY: Cambridge University Press.
• Vermunt, Jeroen K. and Jay Magidson (2000). "Latent class cluster analysis." Chapter B1 in Hagenaars and McCutcheon, eds., Advances in latent class models. Cambridge, UK: Cambridge University Press. Related to Latent Gold software, discussed above.