Log-Linear, Logit, and Probit Models

Overview

Log-linear, logit, and probit models are special cases of general linear models (GLM, which includes regression and ANOVA models) to better treat the case of dichotomous and categorical variables.
Log-linear analysis deals with association of categorical or grouped data, looking at all levels of possible main and interaction effects, comparing this saturated model with reduced models, with the primary purpose being to find the most parsimonious model which can account for cell frequencies in a table. Log-linear analysis is a type of multi-way frequency analysis (MFA) and sometimes log-linear analysis is labeled MFA. While log-linear analysis is a non-dependent procedure for accounting for the distribution of cases in a crosstabulation of categorical variables, three major dependent procedures are closely related.

Logit modeling is similar to log-linear modeling, but explains one or more dependent categorical variables. When there is a dependent categorical variable, however, binary and multinomial logistic regression are more commonly used. Logistic regression is also used when the independents are continuous (forcing continuous variables into categories attenuates correlation and is not recommended). Logit regression yields results equivalent to logistic regression, but with different output options. Many of the same problems can be handled by either logit or logistic regression, though the latter has become more popular among social scientists.

Conditional logit handles matched-pairs and panel data, and data for analyzing choices.

Probit is a variant of logit modeling based on different data assumptions. Where logit and logistic regression are commonly used when the levels of the dependent variable split equally or at least not in a pattern, probit regression may be the more appropriate choice when the categories are assumed to reflect an underlying normal distribution, even if there are just two categories. Both logit and probit usually lead to the same conclusions for the same data.

Log-linear analysis is different from logistic regression in three ways:

The expected distribution of the categorical variables is Poisson, not binomial or multinomial.

The link function is the natural log of the dependent variable, not the logit of the dependent as in logistic regression. (A logit is the natural log of the odds, which is the probability the dependent equals a given value (usually 1, indicating an event has occurred or a trait is present) divided by the probability it does not).

Predictions are estimates of the cell counts in a contingency table, not the logit of y. That is, the cell count is the dependent variable in log-linear analysis.

Loglinear methods also differ from multiple regression by substituting maximum likelihood estimation of a link function of the dependent for regression's use of least squares estimation of the dependent itself. The link function transforms the dependent variable and it is this transform, not the raw variable, which is linearly related to the model (the terms on the right-hand side of the equation). The link function used in log-linear analysis is the log of the dependent, y. The function used in logit is the natural log of the odds ratio. The function used in probit is the inverse of the standard normal cumulative distribution function.
There are several possible purposes for undertaking log-linear modeling, the primary being to determine the most parsimonious model which is not significantly different from the saturated model, which is a model that fully but trivially accounts for the cell frequencies of a table. Log-linear analysis is used to determine if variables are related, to predict the expected frequencies (table cell values) of a dependent variable, the understand the relative importance of different independent variables in predicting a dependent, and to confirm models using a goodness of fit test (the likelihood ratio). Residual analysis can also determine where the model is working best and worst. Often researchers will use hierarchical loglinear analysis (in SPSS, the Model Selection option under Loglinear) for exploratory modeling, then use general loglinear analysis for confirmatory modeling. SPSS supports these related procedures, among others:

Generalized linear modeling. Generalized linear modeling (GZLM), discussed separately, represents a more recent approach for analyzing categorical dependents and independents, thus constituing a different method for implementing log-linear, logit, probit, Poisson regression on cell count data, and other models.

Hierarchical loglinear analysis (HILOG). Select Analyze, Loglinear, Model Selection. Often used for automatic selection of the best hierarchical model.

General loglinear analysis (GENLOG). Select Analyze, Loglinear, General. Often used to refine the best hierarchical model to be more parsimonious by dropping terms.

Poisson regression may be performed within the GENLOG procedure. Often used in event history analysis and other research involving rare events where assumptions of a normally distributed dependent do not apply and the researcher wishes to specify a categorical variable as the dependent variable in a model.

Logit. Select Analyze, Loglinear, Logit. Used when the researcher wishes to specify a categorical variable as the dependent variable in a model (hierarchical and general loglinear analyses are non-dependent procedures).

Conditional logit may be performed under Analyze, Survival, Cox regression. This is a dependent procedure used when observations are correlated rather than independent, as in before-after datasets.

Probit response models. Select Analyze, Regression, Probit. Probit response models are used when the researcher wishes to specify a binary variable as the dependent in a model, and that binary variable is assumed to be a proxy for a true underlying continuous normal distribution. See also the separate section on probit response models, which additionally support logit response

Ordinal regression can also implement logit, probit, and other models, as discussed in the separate section on ordinal regression.

In summary, traditional approaches to categorical data relied on chi-square and other measures of significance to establish if a relationship existed in a table, then employed any of a wide variety of measures of association to come up with a number, usually between 0 and 1, indicating how strong the relationship was. Loglinear methods are similar in function but have the advantage of making it far easier to analyze multi-way tables (more than two categorical variables) and to understand just which values of which variables and which interaction effects are contributing the most to the relationship. For simple two-variable tables, traditional approaches may still be preferred, but for multivariate analysis of three or more categorical variables, log-linear analysis is preferred. Related procedures enumerated above are available when there is a dependent variable, but log-linear analysis itself focuses on explaining cell counts, not explaining the variance in a dependent variable.

Contents

Key concepts and terms
Brief SPSS example
Brief SAS example
Residual analysis
Effect size measures
Hierarchical loglinear modeling
General loglinear modeling
Logit regression
Conditional logit
Probit
Other logit and probit models
Tobit
Poisson regression
Assumptions
Frequently asked questions
Bibliography

Key Concepts and Terms

Types of variables. In log-linear analysis, variables may be factors, dependents, covariates, contrasts, or cell structure variables. Not each type of log-linear procedure supports all types.
1. Factors, also known as "response variables," are categorical variables which define rows and columns in a crosstabulation table.In hierarchical loglinear models in SPSS, 10 factors are possible and the researcher can specify which levels of each to analyse. In probit models, the factor's levels are used to define subgroups for analysis. Factors are input in the main log-linear dialog for general, hierarchical, logit, or probit models.
2. Dependents pertain specifically to logit models, where one or more categorical variables may be specified as dependent. Probit models allow entry of a variable either as a response frequency variable (a variable with counts of individuals who responded when exposed to a particular stimulus level or stimuli levels) and a total observed variable (a variable containing the total number of individuals exposed to a particular stimulus level or combination of stimulus levels.)
3. Covariates. These are continuous variables and the the mean covariate values of cases in any given cell are used to model cell frequencies. Usually covariates are conceptualized as confounding factors or control variables. General, logit, and probit models allow covariates but not hierarchical log-linear analysis.
4. Cell structure variables are used to weight cells within the table. This is used for a variety of purposes, including forcing structural zeros (discussed in the FAQ section) and table standardization. Cell structure is an option in general and logit models, and hierarchical models has a "cell weight" variable entry option. The probit procedure does not support cell structure/weights.
5. Contrast variables. You can specify a set of contrast variables to test the differences between model effects by computing generalized log-odds ratios. The values of the contrast variable are the coefficients for the linear combination of the logs of the expected cell counts. Contrast variables may be entered in general, hierarchical, and logit log-linear models, but not as such in probit models.
Log-linear modeling is an analog to multiple regression for categorical variables. When used in contrast to log-linear regression models like logit and logistic regression, log-linear modeling refers to analysis of table frequencies without necessarily specifying a dependent. Rather the focus is in accounting for the observed distribution of cases.
- Saturated models and effects. A saturated log-linear model is one which incorporates all possible effects: a 1-way effect for each variable, all 2-way interaction effects for models with two or more variables, all 3-way interaction effects for models with three or more variables, and so on. Overall, there will be (2^k - 1) terms plus a constant in a saturated model's equation predicting the log of an expected table frequency, where k is the number of variables. A saturated model imposes no constraints on the data and always reproduces the observed cell frequencies. As such the saturated model forms the "baseline" for log-linear analysis. The researcher tries to see if a model restricted to only some of the possible effects can explain the observed frequencies. In the SPSS, saturated models are obtained if one does not check Custom under the Model button and thus does not enter specific main and/or interaction effects; there is a Saturated model choice, which is selected by default, and this is true of general, hierarchical, and logit loglinear analysis.
- Parsimonious models. Saturated models always have perfect goodness of fit to the data, but this is a trivial finding. The purpose of log-linear modeling is to eliminate some of the effects while still being able to achieve goodness of fit. A parsimonious model is the most incomplete model which still achieves a satisfactory level of goodness of fit. To put it another way, the researcher tests to see if a restricted model does not significantly differ from the saturated model. If there is no significant difference, then the researcher concludes that the effects dropped from the saturated model were not needed to explain the observed distribution of data in the table. The researcher explores in this manner until the most parsimonious model which still has acceptable fit is found. In SPSS, parsimonious models are obtained by checking Custom under the Model button and entering specific main and/or interaction effects as a design (or using the /DESIGN command with parameters in syntax mode; with no parameters, a saturated model is specified).
  As elaborated below in the section on effects, a saturated log-linear model takes the form: the natural log of the frequency for any cell equals the grand mean (the constant) plus the sum of the lambda parameter estimates for all 1-way, 2-way, 3-way, .... k-way interaction effects in a model with k variables. Depicted graphically, a saturated model with six variables (A through F) would show connecting lines from each variable to each other variable, for a total of (2^k - 1) = 127 effects and 16 lines. However, a parsimonious model such as that below might have far fewer connecting lines.
  
  The parsimonious model above has the form: ln(cell frequency) = + A + B + C + D + E + F + A *B + B*C + B*D + C*D + D*E + E*F + B*C*D, for a total of only 13 effects and 6 connecting lines. Each effect is reflected in a parameter estimate, discussed below.
- Effects as categorical control variables.. In the traditional elaboration model using crosstabulation, one divided an original table (ex., literacy by race) into two or more subtables based on the control variable (ex., subtables for region = South or North). For instance, the overall table might show an association between race=black and literacy=not, but if the South tended to have more blacks than the North and more non-literates for both races, then the original relation might well be spurious because region was a control variable. In general, the control variable subtables might show the same relationship as the original table, no relationship, or a relationship for one subtable but not the other. This is illustrated below for a small sample for purposes of showing the corresponding parsimonious model effects when analyzing the same data using loglinear analysis:
  
  Original Table
  - Black White
  
  Not Literate 6 2
  
  Literate 2 6
  
  South
  - Black White
  
  Not Literate 3 1
  
  Literate 1 3
  
  North
  - Black White
  
  Not Literate 3 1
  
  Literate 1 3
  
  Model A: Race*Literacy Effect
  
  South
  - Black White
  
  Not Literate 6 2
  
  Literate 0 0
  
  North
  - Black White
  
  Not Literate 0 0
  
  Literate 2 6
  
  Model B: Race*Literacy + Region*Literacy Effects
  
  South
  - Black White
  
  Not Literate 2 2
  
  Literate 2 2
  
  North
  - Black White
  
  Not Literate 4 0
  
  Literate 0 4
  
  Model C: Region*Race*Literacy Effect
  
  The original table above is shown with three different possible splits by the control variable Region.
  - In Model A, the split tables have the same relationship as the original table. There is no control effect of Region and therefore Region is not part of the loglinear generating class reported by hierarchical loglinear modeling using backward elimination. That is, Race*Literacy accounts for the data found in the table.
  - In Model B, there is full explanation (total control by the control variable Region) and the final loglinear generating class needed to account for the data in the table is the two two-way interactions of Race*Literacy plus Region*Literacy, since both variables affect Literacy.
  - In Model C, the original relationship disappears (is controlled) in the South region but is stronger than the original in the North region, showing the original table to be a misleading average. For Model C the loglinear generating class is the three-way interaction of all variables: Region*Race*Literacy.
- Convergence. In loglinear modeling, parameter estimates are calculated by an iterative process. SPSS's output log reports how many iterations were taken to reach a solution (convergence). If the model did not converge, the researcher could try increasing the number of iterations allowed. In SPSS, maximum iterations has a default of 20 but can be reset under the Options button. Most models converge under the default setting.
- Goodness of fit is measured by the likelihood ratio, also known as likelihood ratio chi-square, deviance chi-square, L², or G², discussed below. For two-way tables, traditional Pearson chi-square may be used also. Both the likelihood ratio and chi-square are based on assessing the difference between observed cell frequencies and frequencies predicted by the model.
  - Pearson chi-square. For two-way tables, chi-square may be used to test association. A significant chi-square means that one rejects the null hypothesis that the two variables are independent. A significant likelihood ratio means that one rejects the null hypothesis that the researcher's restricted (parsimonious) model does not differ from the trivial saturated model. Testing the "independence model," discussed below, is equivalent to traditional chi-square tests for lack of association between variables.
  - Likelihood ratio. SPSS generates both chi-square and the likelihood ratio. Because the maximum likelihood estimation used in loglinear analysis minimizes a likelihood function, most researchers prefer to use the likelihood ratio rather than Pearson's chi-square as a goodness of fit test. When the likelihood ratio is not significant then the model being tested is a good fit to the data because this means the parsimonious model is not significantly worse than the perfect-fitting saturated model. This is the same as saying that a non-significant likelihood ratio means the model-predicted frequencies are not significantly different from the observed frequencies. The likelihood ratio is printed by SPSS in the "Goodness of fit" table in output. If a stepwise method has been selected, then at the end of each step, SPSS will print out the "Likelihood ratio chi square". Its P value will be >.05 for a well-fitting model. The larger the sample, the closer likelihood ratio chi-square and Pearson chi-square are apt to be.
  - Factor list warning: SPSS computes goodness of fit measures based on the number of cells created by a table based on all the variables in the factor list. This is not necessarily the same as all the variables in the model, if one asks for a "Custom" model and leaves out one or more factors in the factor list. Goodness of fit measures will differ for the same model, depending on variables in the factor list.
  - In the illustration below, loglinear analysis is run for two models, both using data from Example A above. In each case, all three variables (Race, Region, and Literacy) are listed as factors in the inital SPSS general loglinear dialog (Analyze, Loglinear, General). However, in the upper half of the figure below, the model tested is the one-factor independence model: Region + Race*Literacy. A one-factor independence model is one which assumes one factor (region in this example) is unrelated to the others. Under the Model button, the 1-way effect Region and the 2-way effect Race*Literacy is entered. This tests if the Race*Literacy and the main effect of Region account for the cell count in the Example A table. In the lower half of the figure below, a one factor model tests if the cell count may be accounted for by the Race*Literacy interaction alone. Both models are non-significant, meaning they do not differ significantly from the saturated model (all possible effects, which always explains the cell count entirely). Since both models are non-significant, the researcher would choose the model in the lower half of the figure on parsimony grounds, as one would expect given the irrelevance of Region for Example A.
- Crosstabs vs. loglinear analysis. Crosstabs will yield the same chi-square results as loglinear analysis performed on the independence model. The independence model assumes that there are no interactions among the variables. For the Race-Literacy-Region example A above, for instance, the loglinear independence model is simply the main effects without interactions: "Region + Race + Literacy". This means there is no Race*Literacy effect, no Region*Literacy effect, and no Region*Race*Literacy interaction effect of Race and Region on Literacy. The comparison of the crosstabs and the loglinear analyses is illustrated in SPSS output below:
  - Likelihood ratio and Pearson chi-square goodness of fit. The model is non-significant by both tests. The likelihood ratio test is more commonly reported. For loglinear analysis, the p "Sig." value for bothe tests tests the null hypothesis that the researcher's restricted (parsimonious) model involving all three variables does not differ from the trivial saturated model (the model with all possible main and interaction effects). For these data, the finding of non-significance means this model (the independence model with main effects only and no interactions) is not significantly different from the saturated model (all possible effects) which explains 100% of the data distribution in the table. That is, the independence model is a good fit. Warning: Note, however, that very small samples, such as the one in this pedagogical example, may lead to misleading findings of non-significance.

Original Table
-	Black	White
Not Literate	6	2
Literate	2	6

Brief example of how loglinear analysis works in SPSS

Example. Data are analyzed for the variables party, race, and gender, for a sample of 737 people. The table being analyzed is shown below, created under Analyze, Descriptives, Crosstabs. There are seven effects: party, race, gender, party*race, party*gender, race*gender, and party*race*gender. The research problem in loglinear analysis is to determine how few effects can satisfactorily explain the cell count in this table, or if a saturated model (all seven effects) is needed.
1. The saturated model. The saturated model will always explain 100% of the cell count in a table. As such it serves as a baseline. If the researcher can find a more parsimonious model (here, one with fewer than the seven effects in the saturated model)which is not significantly different from the saturated model, the researcher's model has good fit. As shown in the figure belwo, in SPSS, the saturated model may be created by selecting Analyze, Loglinear, General; then party, race, and gender are all entered as factors; then the model may be run accepting the defaults.
  
  As can be seen from the output below, for a saturated model, all residuals (difference between actual and model-expected counts) are 0.0. That is, saturated models display perfect explanation. In the goodness of fit table, no probability levels are shown because they cannot be computed for a saturated model. Rather, the saturated model is the baseline.
2. The independence model. The independence model is one with no interaction effects, only main effects. In this case, the main effects are party, race, and gender. After clicking the Model button in SPSS, "Custom" is checked and the main effects entered as below.
  
  This time the observed and model-expected cell counts differ and the residuals are non-zero. The log likelihood chi-square test is significant, meaning the independence model differs significantly from the all-explaining but trivial saturated model. That is, the independence model is not good fit. The research challenge is now to search for the most parsimonious model between the independence model and the saturated model which is not significantly different in model fit from the saturated model. This search process can be automated in SPSS using hierarchical loglinear modeling (a.k.a. model selection or HILOG), discussed below. For instructional purposes, however, manual search using general loglinear modeling (GENLOG) is utilized. GENLOG is discussed in more detail below.
3. Model dropping the highest level of interaction. One strategy for manual searching is to drop higher level interactions first, then work backwards toward the independence model until fit stops being insignificantly different from the benchline saturated model. For this example, the highest level of interaction is the three-way interaction: party*race*gender. Output below is for this model, which has all main effects and all two-way effects, but no three-way effect. Such a model is sometimes called a "homogenous association model." As can be seen, this model has good fit because it is insignificantly different from the saturated model at p = .954. Further search is still needed, however, to determine if this is the most parsimonious well-fitting model.
4. Conditional independence model. It may be that one or more of the two-way effects might be dropped to achieve a yet more parsimonious well-fitting model. A model with only some of the two-way interactions is a "conditional independence model." The researcher may consider dropping party*race, party*sex, or race*gender. While each of the three might be dropped in turn, it is helpful to look at the parameter estimates table This is not part of SPSS default output but must be requested under the Options button by checking "Estimates". Since the race*gender two-way interaction is the only one of the three two-way interactions for which all level combinations are non-significant, race*gender is the best candidate to drop. (Note parameter non-significance means the parameter is not contributing to the model. This is not to be confused with non-significant goodness of fit, where non-significance corresponds to good fit.)
  
  The goodness of fit for the conditional independence model dropping the race*gender interaction is shown below. Fit is non-significant, which indicates a well-fitting model. As the model is more parsimonious than the model with all two-way interactions, it would be preferred.
5. Other models. This example is discussed in more detail in the section below on general loglinear modeling. Dropping either of the remaining two-way interactions will fail to yield good model fit. One-way (main) effects are subsumed by the two-way interactions in the model, which include all three main effects: if all main effects are dropped under the Model button, one will still get the identical goodness of fit table, with the same degrees of freedom and p-levels. That is, though the design equation in table footnote b will differ, the models will be identical. The researcher concludes that cell count in the original table can be accounted for by the interaction of gender and party and the interaction of race and party, and that the interaction of race by gender is not important to the explanation.

Brief example using SAS.

Example. The same dataset is used as for SPSS above.

SAS syntax. The SAS syntax below implements loglinear analysis parallel to that in the SPSS example above. Explanatory syntax notes within the slash/asterisk symbols are comments not executed by SAS. SAS has many more PROC CATMOD options than illustrated here.

PROC IMPORT OUT= WORK.logit2 
            DATAFILE= "c:\Datasets\logit_table2.sav"
            DBMS=SPSS REPLACE;
RUN;
TITLE "PROC CATMOD LOGLINEAR ANALYSIS EXAMPLE" JUSTIFY=CENTER; /* Optional title on each page */
ODS HTML; /*turn on html output*/
ODS GRAPHICS ON;   /*turn on ods graphics*/
TITLE "PROC CATMOD LOGLINEAR ANALYSIS EXAMPLE" JUSTIFY=CENTER; /* Optional title on each page */
PROC CATMOD DATA=logit2;   /* Use the work data file from PROC IMPORT */
WEIGHT weight; /* The imported dataset has a variable called weight with cell frequencies */
MODEL party*race*gender=_response_
   / ML PROB NORESPONSE NODESIGN NOPROFILE PRED=FREQ;
   /* model the three factors */
   /* PROB generates probability estimates */
   /* ML requests maximum likelihood estimates; or use WLS for weighted least square estimates or GLS for generalize least squares, which is identical to WLS */
   /* NORESPONSE suppresses the _RESPONSE_ matrix  */
   /* NODESIGN suppresses the design matrix */
   /* NOPROFILE suppresses the population and response profiles */
   /* PRED=FREQ */
TITLE "SATURATED MODEL" JUSTIFY=CENTER; 
LOGLIN party|race|gender @3; /* request saturated model (all terms up to 3-way interaction) */
RUN;
TITLE "INDEPENDENCE MODEL" JUSTIFY=CENTER;
/* Note ODS will print only the first title. All titles appear in the output window */
LOGLIN party race gender; /* request independence model */
RUN;
TITLE "HOMOGENOUS ASSOCIATION MODEL" JUSTIFY=CENTER;
LOGLIN party|race|gender @2; /* request all terms up to 2-way interaction */
RUN;
TITLE "CONDITIONAL INDEPENDENCE MODEL" JUSTIFY=CENTER;
LOGLIN party race gender party*race party*gender; /* request conditional independence model dropping race*gender */
RUN;
ODS GRAPHICS OFF;
ODS HTML CLOSE;

SAS output for the saturated model. Compare to SPSS output above. SAS cell counts and residuals not shown for the saturated model but parallel in format those for the independence model below.
SAS output for the independence model. Compare to SPSS output above.
SAS output for the homogenous association model. Compare to SPSS output above.
SAS output for the conditional independence model. Compare to SPSS output above. Note that the previous Anova table for the homogenous association model gave an overall significance level for each effect, with the effect for race*gender being non-significant and therefore a candidate to drop in the conditional independence model. This substitutes for the parameter estimates table in SPSS, discussed above.

Warning: Whereas SPSS will subsume the subsidiary main effects of party, race, and gender and will give the same likelihood ratio test whether or not main effects are entered under the Model dialog when requesting a custom model for the two interaction effects (party*race, party*gender), SAS does not. To obtain results parallel to SPSS, the three main effects must be explicit in the LOGLIN command line of PROC CATMOD. Otherwise the main effects are omitted, giving the following likelihood ratio, which is for the interaction effects in the absence of the main effects.

Residual analysis

Overview. Goodness of fit confirms/disconfirms the overall model, whereas residual analysis helps the researcher spot outlier cells - where the parsimonious model is not fitting as well, even if overall the model is a well-fitting one by the likelihood ratio chi-square test. In a well-fitting model, ideally, residuals should be small, non-significant positive and negative values which are evenly distributed across all the cells of the table. Note that when there are many cells, many standardized residuals are generated and one would expect 5 percent of the cells to have significant residuals simply due to chance: typically these are not treated as outliers.
- Residuals depend on the model. Consider the initial small-sample example modeling region, race, and literacy. As the figure below illustrates, residuals depend upon the model being tested. For the independence model for Model A in the earlier example, the table below shows residuals corresponding to a crosstabs chi-square test, which is also a test of independence. For the Race*Literacy model shown in the lower half of the figure, however, all residuals are zero because for Model A, there is no interaction with region and the count in the table is explained fully by thr Race*Literacy interaction (and its subsidiary main effects, Race and Literacy).
- Residuals of the most parsimonious model. Once the most parsimonious model is selected, SPSS can compute the expected frequencies. Unlike the simple example above, normally the expected values will differ from the actual values. The expected frequencies can be subtracted from the observed cell frequencies to give the residuals. The smaller the residual, the better the model is working for that cell. Likewise, large residuals indicate marginal (row and column) conditions where the model is not working well. SPSS shows residuals in a table of "Cell Counts and Residuals." Plots are selected in SPSS from the Options dialog, illustrated below.
  1. Adjusted residuals plot. A residual is the observed minus expected frequency, a.k.a. "raw residual." The raw residual divided by the estimated standard deviation of observed counts is the "standardized residual." Significant standardized residuals have values > 1.96 and such cells may be considered "model outliers." As a rule of thumb, more than one model outlier per 20 table cells may cause the researcher to seek a different model. The standardized residual divided by the estimated standard error. This gives residuals with a mean of 0 and a standard deviation of 1. "Adjusted residuals" penalize standardized residuals for the fact that large expecteds tend to have larger residuals. Looking at the cells with the largest adjusted residuals shows where the model is working least well. The adjusted residual is also called the studentized residual. Adjusted residuals are preferred over the standardized residuals when assessing normality. Checking "Adjusted residuals" in SPSS generates this plot for the earlier SPSS and SAS examples using Party, Race, and Gender.
    
    Discussion: Each cell in the matrix above has 18 dots because for this example factor space has 18 cells (3 party * 3 race * 2 gender; dots may overlap). That the observed by expected counts plots in the matrix form a 45-degree line indicates a well-fitting model. For the plots involving adjusted residuals, a random cloud (no pattern) is desirable. For these data there is no linear trend for residuals to increase or decline as expected or observed count increases, but there is greater variance of residuals for lower counts.
  2. Normal probability (Q-Q) plots. Checking "Normal probability for adjusted" in SPSS Options generates these two plots. In a well-fitting model, residuals will be normally distributed, with most residuals near 0 and then trailing off in a bell-shaped curve of too-high and too-low values. The Q-Q plot of expected normal values and adjusted residuals should approximate a 45-degree angle when residuals are normally distributed. For "detrended Q-Q plots" there should be no trend in how adjusted residuals deviate from a normal distribution (cases will be scattered randomly with respect the the horizontal 0 line). Below, residuals deviate slightly from normal, but probably would be considered to be within acceptable range.
  3. Deviance residual plot. Checking "Deviance residuals" in SPSS Options generates this matrix plot. The deviance residual, also called the "studentized deviance residual," is a more accurate version of the adjusted residual. It is how much a cell contributes to the likelihood ratio. Likelihood ratio chi-square equals the sum of squared deviances. It also has a mean of 0 and a standard deviation of 1 for large samples. The matrix is interpreted similarly to that for adjusted residuals above.
  4. Normal probability (Q-Q) plots for deviance. If the researcher checks the "Normal probability for deviance" option under the Options button in SPSS, two plots will be output (not shown here) which parallel the normal probability plots discussed above, but using deviance residuals.

Effect size measures

Overview. A saturated model has all possible effects and the strategy of log-linear analysis is to set as many of these effects to zero as possible in order to find the most parsimonious model. A one-way effect is the influence of a single row or column variable on cell counts (ex., gender on vote). A two-way effect is the interaction (joint) effect of a column and a row variable (ex., gender and income category on vote). A significant two-way effect means the two variables are related (are not statistically independent of one another). For three variables, there can be one 3-way effect, three 2-way effects,. and three 1-way effects. In general, for k variables, there are a maximum of (2^k - 1) effects. As in regression, a loglinear model also has a constant, which reflects the cell count when all other effects are zero. Finally, it must be emphasized that as in all multivariate procedures, effect sizes are "controlling for other variables in the model."
- Parameter estimates. In loglinear analysis, parameter estimates are log-odds ratios, which are effect size measures. Log-odds means log of the odds ratio, so the odds ratio for any estimate, b, is EXP(b), which is the natural log base e to the power of b. Log-odds are not "b" coefficients (not slopes) as are regression estimates. To make this distinction, loglinear parameter estimates are usually symbolized by lambda (), not b (though some authors do use b). Mu () is the usual designation for the constant. These coefficients are obtained in SPSS by requesting Estimates under Options in the loglinear dialog. The parameter estimates table previously discussed with regard to the brief SPSS and SAS loglinear model for Party, Race, and Gender is reproduced below. (See also a second example discussed below).
  - Standardized parameter estimates (Z scores) in column "Z" of the parameter estimates table above may be used to assess which levels of which categorical variables in the model are most or least important. Z scores are the estimates divided by their standard errors. In the table above, for instance, the interaction of party=1 (Democratic) and race=1 (White) is consequential and negative (being White reduces odds of being Democratic). In contrast, the interaction of party=2 (Independent) and race=1 (White) is small and if not for being nonsignificant would be considered positive (being White increases the odds of being Independent). In general, the more positive (if significant) the parameter estimate for an effect, the more observations are predicted to be in a cell over and beyond those predicted by the constant and other effects. The more negative (if significant), the fewer observations are predicted. If the parameter estimate is non-significant, the effect is not associated with any change in cell frequencies which are predicted by the constant and other effects.
  - Model equations in loglinear analysis. The loglinear model is one in which the natural log of the frequency for any cell is equal to a grand mean (the constant, mu) plus the lambda parameter estimate for the effect of the first independent, plus the lambda for each other independent, plus the lambdas for all 2-way, 3-way, or higher interaction effects, according to the number of independents. Thus for two categorical variables, A and B, the saturated model is:
    Ln(F_ij) = + _i^A + _j^B + _ij^AB
    where F_ij is the cell frequency for row i and column j, is the constant, _i^A is the main effect for variable A (the row effect), _j^B is the main effect for variable B (the column effect), and _ij^AB is the interaction effect of A with B. For a two-factor model, this is the saturated model, which is always fully predictive of the table frequencies, but trivial. The the research purpose in loglinear modeling is to determine how many lambdas can be constrained to 0 and still have acceptable estimates of the frequencies.
  - Predicted frequencies. The predicted frequency of any cell is EXP(mu + lambda_1...n), where mu is the constant, lambda_1...n is the sum of the lambdas for all effects in the model, and the EXP() function returns the base of the natural logarithm (e = 2.71828182845904) raised to the power of the sum of mu plus the lambdas. Note the lambdas are added to the constant: they are not like b regression coefficients, which must be multiplied by their respective variable values for a case when making a prediction. This is illustrated using parameter estimates from a loglinear analysis of the table below, in which Race=0 is African-American and Literacy=0 is "Not literate." The table was generated in SPSS using Analyze, Descriptives, Crosstabs. The "expected value" is that for chi-square, which takes account of the main effects of race and literacy but not the race*literacy interaction effect. For the Race=0, Literacy=0 cell (whose count determines the three others in a 2x2 table) it can be observed that the actual count is 6 and the count expected by main effects alone is 4.
    
    Computing predicted frequencies is illustrated in the spreadsheet figure below, which uses the parameter estimates from a loglinear analysis of the table above. Analyze, Loglinear, General was selected and a saturated model requested. The saturated model fully explains the cell count in the table. The three effects in a saturated model for these data are Race, Literacy, and Race*Literacy. Adding the constant (mu) and all the estimates (the lambdas) gives a sum of 1.792. Since in loglinear analysis, estimates are the natural log of the cell counts, one can compute the actual cell count as the natural logarithm base e exponentiated to the power of the estimate. In SPSS, spreadsheets, and most software, this is EXP(b), where b in this case is mu (constant) plus the sum of relevant lambdas (estimates). EXP(1.792) = 6, so the cell count predicted by loglinear analysis is identical the the actual count and the residual is 0.0. This is always the case for all cells in analysis of a saturated model.
    - Caveat: Delta. By default, SPSS adds a factor, called delta, to each cell to assure no cell has zero frequency. By default, delta=.5. For large samples delta matters little. For small samples it can made a difference. For the small sample above, accepting delta=.5 will lead to a cell count estimate of 6.5. To get the actual value (6.0) it is necessary to reset delta to 0.0 as illustrated below and as done in the example above.
- Odds ratios. As state above, the odds ratio is EXP(lambda) and is a type of effect size measure.
  - An odds ratio of 1.0 means the variable or effect has no effect on the cell count in loglinear analysis or is statistically independent of the dependent in logit or logistic regression.
  - An odds ratio above 1 indicates that the variable or effect increases the likelihood of the event. The "event" depends on the procedure, with the event being cell count in loglinear analysis or the dependent equaling a certain value in logit or logistic regression. For the latter, the event depends how the researcher has chosen to code the categorical dependent variable. By default, binomial logistic regression, for instance, uses indicator coding (that is, 0, 1) and the "1" value indicates the event of interest. That is, a unit increase in the independent variable is associated with an increase in the odds that the dependent equals 1 in binomial logistic regression. For multinomial logistic regression, the "event" is usually the highest value by default. Put another way, an odds ratio > 1.0 means increasing the given variable increases the odds that the count will be increased in loglinear analysis or that the dependent variable equals a given value (usually 1 or other highest value) in logit or logistic regression, by a factor of the odds ratio. For example, an odds ratio of 2.0 corresponds to a factor of 2, which might mean the odds increase from 1:1 to 2:1. The larger the positive difference between the observed odds ratio and 1.0, the stronger the relationship.
  - An odds ratio below 1 indicates that the independent ariable decreases the likelihood of the event.
  - Second example. Consider a log-linear model (where cell count is predicted, not a dependent variable) for a crosstabulation involving gender, party, and race, illustrated below. Note that although the same variables are modeled, the previously discussed model above was for the saturated model, not the more parsimonious model here.
    - Coding and reference categories. For the data above, Party is coded 1=Democrat, 2=Independent, 3=Republican. Gender is coded 1=Male, 2=Female. Race is coded 1=White, 2=Hispanic, 3=Black.
    - Standardized parameter estimates. Here it can be seen that the (Party=3)*(Race=2) interaction, which corresponds to Republican Hispanics, is not significant. The (Party=2)*(Race=2) interaction is also non-significant. All of the other combinations of interacting values significantly contribute to the explanation of the distribution of data in the table. The highest significant Z value for this example is 8.38, for Female Democrats. If we were to go back to the original cell counts and compare the expected cell counts, we would find that the sum of absolute residuals (observed minus expected) for Republican Hispanics was only 3.5, whereas the corresponding residual sum for Female Democrats was 38.5 - far larger. That is, in terms of moving cell counts away from the distribution expected by chance, Female Democrats have a large effect and Republican Hispanics have a low (and non-significant) effect. In general, the ratio of the absolute magnitudes of the standardized parameter estimates (labeled 'Z' in the Parameter Estimates table above) for any two cells reflects the relative importance of those parameters in explaining the frequencies in the table. Standardized parameters are parameters divided by their standard errors and are shown in the Z column in the SPSS output table above.
      - Comparing effect sizes. One-way effects with large standardized parameters ( the Z column) flag the variable values which were most important in explaining the distribution of the data. For one-way effects with three levels or more, the ratio of standardized lambdas will indicate which values contributed the most to deviation from the expected (flat) distribution for that one-way effect (that variable). In the table below (for a different model which includes one-way effects), for instance, where Party = 1 for Democrat, 2 for Independent, and 3 for Republican, Republican is the omitted reference category, and being Democrat is roughly twice as important and in the opposite direction from being Independent, in explaining deviation from a flat distribution of Party. Two-way effects with large standardized parameters flag the most important two-way interactions, and so on for higher-way effects. For any given two-way effect, there will be a parameter estimate for each cell and the ratio of standardized lambdas (the Z column) indicate which cells contributed the most to that effect. Higher-way effects are interpreted analogously to two-way effects.
    - Odds ratios. In the previous table above, the two-way effect [Party=1]*[Gender=1] has a parameter estimate of 1.329, Since e^1.329 = exp(1.329) = 3.777, the odds ratio is 3.777. This odds ratio is a measure of effect size for the Male*Democrat effect. As above, Female Democrats have the highest odds ratio (exp(1.49) = 4.437), while Hispanic Republicans have an odds ratio only a little above 1.0, which signifies weak effect (thus exp(.142) = 1.15). However, for loglinear analysis, standardized parameter estimates (Z-scores) are usually preferred over odds ratios for comparing effects. Accordingly, SPSS prints the Z value but not the EXP(lambda) odds ratio value in this table.
    - Reference categories. Note that there is no coefficient shown for the reference ("aliased") categories, which by default are the highest categories.
    - Confidence interval on the odds ratio. If the 95% confidence interval on the odds ratio includes the value of 1.0, by convention the variable is not considered a useful predictor variable. When the odds ratio is 1, then a change in value of the independent variable is not associated with a change in the odds that the dependent variable equals a given value.
    - Nice properties of odds ratios as measures of strength of relationship. Odds ratios can handle categorical data, not requiring interval data. Odds ratios also do not require variables to be normally distributed or that relationships between variables be homoscedastic (the same along the entire length of the relationship). That is, using odds ratios as measures of strength involves much more relaxed data assumptions than using beta weights as effect size measures in OLS regression. Logistic regression does not use OLS but rather computes coefficients using maximum likelihood estimation (ML), meaning coefficients are calculated which maximize the odds that a dependent variable equals a given value (ex, that party = D). ML estimates benefit from the relaxed data assumptions mentioned above and are now more popular where appropriate.
    - Relative risk. Odds ratios are also computed when requesting the risk coefficient in crosstabs in SPSS, as illustrated below. For further discussion see the section on relative risk and odds ratios as measures of association. Significance testing of odds ratios is discussed in the section on relative risk. In SPSS, the table is obtained by selecting Analyze, Descriptive Statistics, Crosstabs, and then checking Risk under Statistics.
- Partial chi-square is also computed in SPSS hierarchical log linear analysis (HILOG, not GENLOG), discussed next. Partial chi-square can be used to further refine model parsimony. It may be, for instance, that not all 2-way effects need be modeled even though the likelihood ratio test showed 2-way effects should be retained. Two-way effects with non-significant partial chi-squares in the "Tests of Partial Associations" table may be dropped from the model. Note that because the likelihood ratio test takes precedence, higher-order effects with significant partial chi-squares are not included when these higher-order effects fail the likelihood ratio test (that is, if a three-way effect, for instance, has a significant partial association, it will still not be included in the parsimonious model if the likelihood ratio test found that 3-way associations were not significant). Note that lower-order effects which are part of a significant higher-order effect must be retained even if they fail the partial chi-square test (ex., a first-order effect such as Gender might have a non-significant partial chi-square but would be retained if the Gender*Role second-order effect was significant).

Hierarchical loglinear analysis

Hierarchical loglinear analysis

SPSS. Hierarchical loglinear analysis is invoked in SPSS by selecting Analyze, Loglinear, Model Selection from the menus. After the factors are entered (ex., Race, Party, Gender) one must click the Define Range button to specify, for instance, that Party ranges from 0 to 2 (0=Democrat, 1=Independent, 2=Republican). Based on the factors the researcher enters, HILOG automatically creates the highest generating class of interactions and all subsidiary ones (ex., Race*Party*Gender and all subsidary 2-way interactions and the three main effects). To ask for partial chi-squares, click the Options button and check the "Association Table" box (note this is available only when running the saturated model, not when you have clicked Custom and are asking for a parsimonious model).
The "Cell Counts and Residuals" table below is output from HILOG for the party-race-sex data discussed previously above. Delta is set to 0 (the SPSS default adding .5 to all cells is overridden). As can be seen, the saturated model explains cell frequencies perfectly, with 0 residuals.
Generating class refers to the hierarchical nature of HILOG models, which automatically generate a saturated model with all possible effects. Specifically, the generating class is the set of highest-order terms in which factors appear. The hierarchical model contains the terms that define the generating class and all lower-order terms. Thus, for three factors, the generating class factor1*factor2*factor3 would cause SPSS to create a hierarchical model with this three-way interaction plus all subsidiary two-way interactions plus the three main effects of these three factors. For the example above, the highest generating class would be Gender*Race*Party.
Parameter estimates are used by HILOG's algorithms to compute the expected cell frequencies implied by the model. As this is done automatically, in practice the "Parameter Estimates" table may well be largely ignored by the researcher. The parameter estimates which HILOG computes only for the saturated model, differ compared to those for the saturated model in GENLOG and must be interpreted differently (though if the researcher is using HILOG to select a best parsimonious model, there may be no reason to interpret parameter estimates anyway). In HILOG, estimates are deviation contrasts while in GENLOG they are based on indicator coding. Deviation contrasts are coefficients which estimate deviations of each category except the left-out category from the unweighted grand mean of all of the categories of a factor or interaction. The effect of deviation contrasts is to constrain parameter estimates so they must add to 0 for any given main or interaction effect.
- Estimates and coding. HILOG output lists each main and interaction effect in the hierarchy of all effects generated by the highest-order interaction in the set of factors the researcher enters. For main effects, it will number the parameters 1....(p-1), where (p-1) is the number of categories minus the one left-out category. Warning: thus even if the variable Race is coded 0=White, 1 = Hispanic, 2= Black, Parameter 1 will be for White, 2 for Hispanic, and Black will be the left-out reference category. This not-printed parameter estimate for the left-out category is the negative of the sum of the printed parameter estimates (since the estimates must add to 0). Thus in the example below, the left-out category is Black, whose parameter is not shown but is -(.069 + .005) = -.074.
  
  In the "Parameter Estimates" table above, Gender has two categories while Race and Party have three. This is why the main effects for Gender, Race, and Party in the table above have 1, 2, and 2 parameters respectively, with the last in each case being the redundant reference category. The two-way interactions involving Gender thus have 1*2 = 2 parameters, while the Race*Party interaction has 2*2 = 4 parameters. The three way interaction has 1*2*2 = 4 parameters also.
- Parameter numbering for interaction terms. For interaction effects, SPSS numbers the parameters starting with the first value of the first factor entered, then cycles the values of the second one (leaving out the reference categories), etc. First factors cycle slowest; later factors are cycled the fastest. Thus if factor1 has three categories and factor2 has two categories, there will be (3-1)*(2-1) = 2 parameters. Parameter 1 will be for the interaction factor1=1*factor2=1; parameter 2 will be for the interaction factor1=2*factor2=1. To compute the not-printed parameter estimates for the left-out categories, create a table of all possible estimates, then use the sum-to-zero rule to fill in the last row and last column. In the example above, Parameter 1 for Gender*Party = -.160, and this is the parameter for Gender=0*Party=0 (since the first category was coded 0 for each factor). Parameter 2 for Gender*Party = -.013, and this is the parameter for Gender=0*Party=1. The parameters for the remaining combinations of Gender*Party are redundant and not shown, but could be computed using the table method just described.
- HILOG vs. GENLOG parameter coding. In contrast to HILOG's use of deviation coding, with the indicator coding used by GENLOG, the left-out category has a parameter of 0. Indicator coding is like dummy coding in linear regression, whereas deviation coding is like effects coding in linear regression. Re-run in GENLOG the best model selected under HILOG to get the more widely used indicator coding parameter estimates, which GENLOG will give for all models, not just the saturated model.
Tests of k-way effects. The HILOG procedure outputs two tables of k-way tests: (1) Tests that K-way and higher order effects are zero; and (2) Tests that K-way effects are zero. Both rely on the significance of differences in the likelihood ratio (or of Pearson chi-square, which is also shown) when a set of terms of a given order are added to the model. "Order" is 1 for main effects (for ex., factor1, factor2), 2 for two-way or 2nd order effects(ex., factor1*factor2), etc. These tables give an initial idea of what order(s) of effects are or are not appropriate for the most parsimonious model.
Tests of Partial Associations. Tests of partial association (aka, partial chi-square) are partial likelihood ratio (or partial Pearson chi-square) tests and are based on the difference in likelihood ratio (or Pearsonian) chi-square for the model with and without a given term. The "Tests of Partial Association" table in HILOG output is used to test individual terms in the model. Even though the k-way tests may suggest dropping a whole class of terms (ex., all 3rd-order terms), it may be that one or more individual terms in that class are significant. Likewise, the k-way test may suggest retaining a class of terms, yet one or more individual items in that class may be nonsignificant. Keep in mind, though, that regardless of the partial association test, one must retain even non-significant lower-order terms if they are components of a significant higher-order term which is to be retained in the hierarchical model. Thus in the example below, one would retain Gender and Race even though they are non-significant because they are terms in the two significant two-way interactions, Gender*Party and Race*Party. Thus the partial associations test suggest dropping only the Gender*Race interaction.
Backward elimination. If "Use backward elimination" is checked (it is the default), the hierarchical loglinear procedure will automatically screen all possible models in a generating class hierarchy for the most parsimonious one. Starting with the saturation model is not required. The researcher can still enter a custom model using the Model button. Whatever model the researcher starts with, the backward elimination algorithm will drop the least useful term one step at a time, stopping when the deleted effect significance is significant.
As noted earlier, a significant deleted effect "Sig." p value means dropping that term would significantly reduce the model log likelihood chi-square value. The lower the model chi-square value, the more toward a finding of significance, which means the model would be significantly different from the saturated model which explains all of the cell count. That is, lower model chi-square corresponds to poorer model fit. Thus the researcher does not want to drop terms which have significant (p <=.05) deleted effects.

In the example above, hierarchical loglinear algorithm started with the default saturation model in Step 0: Gender*Race*Party, which includes all hierarchically subsidiary two-way (2nd order) interactions and one-way (1st order) main effect terms. In Step 0, the backward elimination algorithm tests to see if the highest order (here, 3rd order) term may be dropped from the model as non-significant. At Sig. = .953, it is indeed non-significant and is dropped, leading to Step 1. Step 1 is the model with all 2nd order (two-way) terms (implying all subsidiary 1st order terms). Since here three factors corresponds to three two-way interactions, each of the three is tested for possible dropping. It is found that Gender*Race is the most non-significant and may be dropped. Thus in Step 2, Gender*Race is dropped and the remaining two 2nd order interactions are used as the generating class. This time no terms are found suitable for dropping (none are found to have non-significant deleted effects). Step 3, the final step, merely lists the generating class for the most parsimonious hierarchical model.
How it works: the backward elimination option calculates partial chi-square for every term in the generating class. Backward elimination deletes any term with a zero partial chi-square , then it sees which effect has the largest significance of change in chi-square if it is deleted (the default alpha significance level is .05). This gives a new model and a new generating class, which is tested in turn. The process continues until there is no significant gain in deleting further terms.
In the final step output under backward elimination, SPSS will print the model likelihood ratio chi square and its significance. This is not the deleted effect chi-square but rather the chi-square for the model as a whole. A non-significant likelihood ratio indicates a good fit, as is the case in this example. Keep in mind that in a hierarchical model, a higher-order term like factor2*factor3*factor4 includes subsidiary 2-way and 1-way effects such as factor2*factor3. If when the researcher goes back to GENLOG to enter a custom model, the researcher would enter the hierarchically-implied terms as well as the actual "final model" terms listed in the HILOG (hierarchical loglinear) output. Of course, backward elimination does not guarantee the most parsimonious well-fitting model - researcher experimentation may still be called for. If one enters the example data into GENLOG (general loglinear modeling, discussed below) and asks for the best model emerging from HILOG (the "Model Selection" option in SPSS), one will get the goodness-of-fit table below, which has the same likelihood ratio goodness of fit as shown in the backward elimination table in HILOG (Sig.=.964, where non-significance corresponds to a well-fitting model). For more on computation of Pearson and likelihood ratio chi-square, click here.

General Loglinear Modeling

Overview. General loglinear modeling is the usual procedure to select when the researcher wishes to use loglinear analysis on categorical variables and wishes to test a model (common models are discussed below) determined in advance (ex., determined by hierarichal loglinear analysis) or wishes to search manually among a finite set of models to determine the most parsimonious one. There are three general approaches:
1. Theory. Ideally, theory and the research literature would lead the researcher to the specification of which effects should be included in the model to be tested.
2. Backward elimination. The researcher starts with the saturated model, which is always 100% well fitting, and deletes terms one at a time, starting with higher order interaction terms, until the likelihood ratio test shows the nested model is significantly different from the saturated one. If the researcher has already used hierarchical loglinear analysis, with its built-in backward elimination process, to arrive at a hierarchical parsimonious model, then in general loglinear analysis the researcher drops terms to see if there is an even more parsimonious non-hierarchical model.
3. Forward aggregation. Though uncommon, it is also possible for the researcher to start with the independence model (only main effects, no interaction effects) and add terms one at a time until the model is not significantly different from the saturated one by the likelihood ratio test.
SPSS. In SPSS, select Analyze, Loglinear, General to select the GENLOG procedure, illustrated below. In the General Loglinear Analysis dialog box, move all the categorical variables of interest (ex., gender, race, and party in the tabled example below) to the Factors box. Clicking OK enters the saturated model by default. (If you click the Models button, you will see that "Saturated model" is checked by default.) There is also an Options button where you may check Frequencies, Residuals, Estimates, Criteria, plots, and more. You will normally want to select at least Estimates, which also gives significance of the estimates for each effect. Under Options, Criteria, you can set Delta=0 to suppress the default under which .5 is added to all cells to avoid having cells with zero count. Set the data distribution assumption (see below), Click Continue. OK.
Data distribution assumptions. Note tht at the bottom of the main GENLOG dialog you may select between Poisson and multinomial distribution assumptions. Both usually yield the same parameter estimates, but the interpretation is different. Poisson is the default. A Poisson distribution is used if total sample size is not fixed (inference is independent of sample size) and the cell frequencies are independent (predicting count in one cell is not affected by counts in other cells). A multinomial distribution, in contrast, is used in the atypical situation where the total sample size is fixed in advance (inference is dependent on sample size) and the cell frequencies are not independent predicting count in one cell is affected by counts in other cells). This would happen in research designs where the sample size is a predetermined n, and the sample is stratified by row and/or column, meaning row and/or column totals are predetermined and sum to n, which in turn means cell frequencies are not independent. Note: When multinomial assumptions are selected, the constant (mu) in the loglinear equation is treated as a given and no confidence limits are computed. This is because the constant in a loglinear model is a function of the means, and the means are determined by the marginals, which are known.
Covariates. In addition to moving categorical variables to the Factors box, interval-level variables can be added in the covariate box in the GENLOG dialog box. Covariates are usually conceptualized as confounding or control variables. For each cell in the table, SPSS automatically computes the mean values of the covariates and uses this mean value similar to a factor to model the cell counts in the table.

Categorical Log-Linear Models. In exploratory log-linear analysis, one is looking for the most parsimonious model which can explain the distribution of observations in a table formed by categorical variables. However, in confirmatory log-linear analysis, one wishes to test a particular model based on theory. In this section various common possible types of log-linear models are discussed. The list of models is not comprehensive, as log-linear modeling supports testing an abundance of different types of models. To illustrate, consider the table above relating Race (0=white, 1=Hispanic, 2=Black) to Party (0=Democratic, 1=Independent, 2=Republican), controlling for Gender (0=Male, 1=Female):
Looking at the significance of effects obtained by asking for Estimates under the Options button of the general loglinear dialog box, is a prime way of reducing the saturated model, eliminatiing non-significant effects.When dropping effects which are nonsignificant, it is best to drop one effect at a time to be sure lower-order non-significant effects don't become significant when a higher-order non-significant effect is dropped. When two or more effects are nonsignificant, start the reduction process by dropping the highest-order nonsignificant effect first, then proceed by dropping one term at a time on subsequent runs. To specify an unsaturated model, in the loglinear analysis dialog, click Model, Custom, and enter the effect terms you want (ex., race, gender, race*gender, highschool).
1. The saturated model. This is obtained as described above simply by entering all categorical variables and clicking OK in the GENLOG dialog box. The saturated model, illustrated below, always explains the table's data distribution fully, though this is a trivial result. The saturation model is the baseline against which subsequent models are compared. The saturated model for this example is: Design: Constant + Gender + Race + Party + Gender * Race + Gender * Party + Race * Party + Gender * Race * Party. The likelihood ratio and the Pearson chi-square for the saturated model will be 0. It is a perfectly fitting model but "Sig." probability levels are not reported as there is always a 100% probability the saturation model will be well-fitting. For more on computation of Pearson and likelihood ratio chi-square, click here.
2. The complete independence model. This model hypothesizes the factors are unrelated. That is, it hypothesizes that there are only main effects and no interaction effects. In this example, the hypothesis would be that race, party, and gender are all unrelated to each other. To create this and any model other than the saturated model, you must click the Model button in the General Loglinear Analysis dialog box, then click Custom. For the complete independence model, move all three factors (race, party, gender) to the Terms in Model box. Set Delta to 0. Click Continue, OK, to obtain output. The model is: Design: Constant + Gender + Race + Party. For these data, the likelihood ratio is 65.6 and the Pearson chi-square is 64.9. Both are significant at the .000 level. This means the complete independence model is significantly different from the saturated model. This further means that at least some of the terms in the saturated model beyond Gender + Race + Party are necessary to account for the distribution of data in the table. That is, at least one of these additional terms in the saturated model is necessary, but so far we do not know which: Gender * Race + Gender * Party + Race * Party + Gender * Race * Party.
  - Quasi-independence models. One can test to see if the independence model applies to a subset of cells in a table, ignoring other cells (ex., ignoring diagonal cells). This is discussed in the FAQ section below in the section on use of forced structural zeros.
3. One factor independence models. These models hypothesize that one of the factors is unrelated to the others. For example, we could hypothesize that race and party are related, but gender is independent. Click the Model button in the General Loglinear Analysis dialog box, then click Custom. Move Gender over into the Terms in Model box. Then highlight both Race and Party in the Factors & Covariates box and then, with Interactions selected in the Build Terms box, click the right arrow to create the term Race*Party in the Terms in the Model Box. Continue, OK. The model is: Design: Constant + Gender + Race*Party. For the data in this example, the likelihood ratio has a probability of .040 and Pearson's chi-square has a probability of .042. This means that there is a significant difference between this particular one factor independence model and the saturated model, which means this model is not a good fit to the data.
  - There are two other one factor independence models for this example:
    Design: Constant + Race + Gender*Party, likehood ratio significance = .000; this model is also not a good fit to the data.
    Design: Constant + Party +Race*Gender, likehood ratio significance = .000; this model is also not a good fit to the data.
4. Conditional independence models. These models hypothesize that only some of the two-way interaction effects are needed to account for the distribution of data in the table. For this example, there were three two-way interaction effects (Gender * Race + Gender * Party + Race * Party) in the saturated model. For instance, we might hypothesize that Gender*Party and Race*Party were important, but Gender*Race was not significant. This model is: Design: Constant + Gender * Party + Race * Party. In the modeling section, highlight all three variables, select All 2-way Interactions, click the right arrow, then delete the Gender*Race term, leaving the two others. Continue, OK. The significance level of the likelihood ratio for these data for this model is .964. This means this model is not significantly different from the saturated model in accounting for the distribution of data in the table. We accept this conditional independence model as a superior model to the saturated model because it is more parsimonious. Note, however, that other models may also be well-fitting.
  - There are two other one factor independence models for this example:
    Design: Constant + Gender * Race + Race * Party, likehood ratio significance = .013; this model is not a good fit to the data.
    Design: Constant + Gender * Party + Gender * Race, likehood ratio significance = .000; this model is also not a good fit to the data.
  - Comparison with the model generated by HILOG. Recall that the best model generated by the Model Selection procedure was the full factorial model minus the Gender*Race interaction for our example. When entered into the general loglinear procedure, this model generates an identical model fit as illustrated below. The HILOG best model is the hierarchically implied version of the conditional independence model. In GENLOG, the model fit is the same but the estimates of the parameters differ.
  - Parameter estimates for the best model. Looking at the significant parameter estimates, shown in red below, we can analyze the relative importance of different effects in the model. For these data, Party was coded 0, 1, 2 for Democratic, Independent, and Republican respectively. Race was coded 0, 1, 2 for White, Hispanic, Black respectively. Gender was coded 0 = Male, 1 = Female. Interpretation of parameter estimates tables is discussed above. The standardized parameter estimates, labeled Z below, show the relative importance of effects. Here the largest effect is for Race=2*Party=0, which is black Democrats. Thus we can say that the pattern of frequencies shown in the original table reveals the variables not to be independent of each other, but rather the interaction of Party with Race and Party with Gender both help explain why observed frequencies are different from what would be expected on the basis of the marginals, and the effect of black Democrats in particular is important to this difference.
5. The homogenous association model. This model hypothesizes that all of the two-way interactions are needed to account for the data, leaving out all others. The model is: Design: Constant + Gender * Race + Gender * Party + Race * Party. In the modeling area, enter All 2-Way Interactions. The likehood ratio significance for the data in this example is .954, almost as high as for the conditional independence model. It is also a well-fitting model but would be rejected as being not as parsimonious as the conditional independence model above (Design: Constant + Gender * Party + Race * Party). In fact, since it was already known that a conditional independence model was acceptable and the one-factor gender*race model was not, the researcher probably would not have tested the homogenous association model.

Conditional loglinear models

Symmetry models. A symmetry model controls for one of the main effects in a table. For instance, in a table formed by gender, party, and race, we might test to see if party and race have similar distributions (are symmetrical) by gender. The model is : Design = party race party*race, where gender, party, and race are in the factor list. Note that if gender were not in the factor list, we would have a perfectly-fitting saturation model of the 2-way party by race table. If the likelihood ratio goodness-of-fit is not significant, then the symmetry model is a good fit. That is, when we leave out gender as a main effect or any of its interactions with party or race, we can still do a good job of predicting the cell frequencies in the gender*party*race table. This is because when you do a good job predicting the party*race table for one gender, you've done it for the other gender, since the distributions are symmetrical. However, for these data the symmetry model is significant, which means it is not a good fit:
Conditional symmetry models. Also called adjusted models of quasi-symmetry, these models test to see if the symmetry model is also conditional on a control variable. The model for our example is: Design= gender party race party*race, where gender, party, and race constitute the factor list. This uses gender as a control, which is to say that expected counts are constrained so it is as if there were an equal chance of being male or female. Then what is tested is whether in this constrained situation, it is equally probable that a case is in cell(i,,j) as in cell(j,i) - that is, if the distribution of party and race is symmetrical by gender, once frequencies are adjusted to equalize males and females. [Whereas the symmetry model tests if in the unconstrained, observed data, it is equally likely a case would fall in cell(i,j) or in cell(j,i), with no frequency adjustments for totals by gender]. For these data, the conditional symmetry model with gender as control is not a good fit since the likelihood ratio is significant.
- Marginal homogeneity test. To test if marginals are the same among values of a control variable (gender in our example), subtract the goodness-of-fit chi-square for the conditional symmetry model from the chi-square for the symmetry model. Use degrees of freedom = (r-1) or (c-1), whichever is less, where r = number of rows and c= number of columns in the table without the control variable (in our example, the party*race table). If the difference is significant, then the researcher concludes the marginals are not the same for the various categories of the control variable (lack of marginal homogeneity). For these data, the chi-square difference = 16.991 - 16.024 = .967, and df = 3 - 1 = 2. In a chi square table for df = 2 and alpha = .05, the cutoff value is 5.99. Therefore the difference is not significant, leading the researcher to conclude the marginals are the same for the categories of gender.

Ordinal models

Overview. If the row and/or column variable consists of ordered categories (ex., ascending income groups or age cohorts, or simply Likert-type scales) then the category codes may be used to achieve more parsimonious models than if order information were not known.
1. Linear-by-linear association models assume both the row and column variable are ordinal. Let B = the estimated regression coefficient for the product of the category codes for the row and column variables. In a linear-by-linear association model, Design: Constant + B + rowvariable + columnvariable. In SPSS, Transform, Compute and set B = rowvariable*columnvariable; then in Analyze, Loglinear, General, enter rowvariable and columnvariable as Factors and enter B as a Cell covariate; click Model, Custom, and specify a model with Main effects for rowvariable, columnvariable, B.
  Consider the following table, in which TestRank is used to predict WorkRank:
  
  The estimate of the B regression coefficient is shown in the "Parameter Estimates" table, B row, Estimate column.
  
  If the likelihood ratio (or Pearson chi-square) is nonsignificant, there is goodness-of-fit achieved simply by adding the B linear-by-linear association (interaction) effect to the complete independence model (which would be Design: Constant + rowvariable + columnvariable). For these data, a finding of significance means the linear-by-linear interaction terms should not be added to the model.
2. Row-effects models assume only the column variable is ordinal. It is called "row effects" because it adds to the complete independence model the effects of the interaction of the row variable with the ordinal column variable treated as a covariate (rather than entered as a simple rowvariable*columnvariable 2-way interaction). Let C = a copy of the column variable, created in SPSS by Transform, Compute, C = columnvariable (this copy is needed because the column variable can't be both a Factor and a Covariate). In a row effects model, Design: Constant + rowvariable + columnvariable + rowvariable*C. In SPSS, Analyze, Loglinear, General, enter rowvariable and columnvariable as Factors and enter C as a Cell covariate; click Model, Custom, and specify a model with Main effects for rowvariable, columnvariable; and specify an interaction effect for rowvariable*C. The "Parameter Estimates" table will show the row and column main effects, and the effects of the interaction of the row variable with the column variable used as a covariate.
  
  If the likelihood ratio (or Pearson chi-square) is nonsignificant, there is goodness-of-fit. Here that is not the case.
3. Column-effects models assume only the row variable is ordinal. These are identical to row-effects models, except the row variable is copied and used as the covariate.

Logit Loglinear Models/Logit Regression

Logit loglinear analysis

Multinomial logit analysis

multinomial logistic regression

The table being analyzed is defined as the cross-classifications of all the factors and dependent variable as specified on the main "Logit Loglinear Analysis" dialog box, whether or not all factors are modeled (under the Model button). That is, a factor may not be used in a model but if it is listed in the main dialog, it will help define the table whose frequencies are being explained.
Logit models model the proportions of cases (usually people) in each category of the dependent(s) for each category of the independent(s). For instance, in a study of race causing party id, logit models would model the ratio of one party to another for each category of race. However, there could be multiple dependent variables and/or multiple independent variables. Logit regression can be seen as a subclass of general loglinear models, where the MODEL parameter is set to "multinomial" (rather than the usual Poisson) and only the main effect of the dependent(s) plus their interactions with independent(s) are modeled, as discussed below. Logit models normally do not include a constant as the constant cancels out in computation, though SPSS will print estimates for the constant, warning in a footnote that "Constants are not parameters under the multinomial assumption."
Logit models under GENLOG. In SPSS, when one selects Analyze, Loglinear, Logit, one gets the GENLOG module. However, when invoked under the Logit option, GENLOG operates differently than if selected by Analyze, Loglinear, General:
1. There is a BY statement in the factor list, with the dependent(s) before and the independent(s) after the BY.
2. The MODEL will default to MULTINOMIAL.
3. DESIGN will be of the format described below, which includes only estimates for effects involving the dependent variable(s). It does not include main effects for the independent(s) or interactions solely among the independent and/or covariate variables.
4. Though SPSS will list a constant in the design and will compute constants, a footnote reminds the researcher that "Constants are not parameters under the multinomial assumption. " The Model button in SPSS contains a checkbox for "Include constant for dependent" to cause constants to be computed and output. For custom models, the researcher can suppress computation of the constant.
The DESIGN takes these forms for the saturated model:
1. One predictor x1 of the dependent y: DESIGN = y + y*x. This was the design for the output illustrated above for Party predicted by Income.
2. Two predictors x1 and x2 of y: DESIGN = y + y*x1 + y*x2 + y*x1*x2
3. When there are two or more dependents, the logit model can become complex, but the design still is limited to terms involving the dependent variables and does not include main effects for the independent variables nor interactions of just the independent variables. Logit loglinear design for two dependents and two independents, for the saturation model: dependent1 + dependent2 + dependent1 * dependent2 + dependent1 * independent1 + dependent1 * independent2 + dependent2 * independent1 + dependent2 * independent2 + dependent1 * dependent2 * independent1 + dependent1 * dependent2 * independent2 + dependent1 * independent1 * independent2 + dependent2 * independent1 * independent2 + dependent1 * dependent2 * independent1 * independent2.
For unsaturated models, obtained under the Custom choice under the Model button, the design will include the main effect of the dependent plus the effect of the dependent interacting with whatever terms ar listed. If x1 is listed, the design will be y + y*x1. If x1 and x2 are listed, the design will be y + y*x1 +y*x2. Etc.

Thus in the example above, Gender and Race are used to predict Party. The model is not saturated since Gender*Race is not modeled. The logit model thus includes Party (the main effect of the dependent) plus Party*Gender (the dependent's interaction with the first factor) plus Party*Race (the dependent's interaction with the second factor). Because the model is not saturated, it is possible for residuals to differ from 0 and for the goodness of fit to be computed. Because model fit is non-significant in this example, the model is considered well-fitting.
Logits (parameter estimates) in logit loglinear models. Logit models deal with the odds that a person in a given cell has the dependent = the reference category (by default, the highest category). The parameter estimate for y=1, x=1 is the logit for that cell (the first cell in the table, in the upper left). The base of the natural log e to the power of the logit [exp(logit)] expresses the effect relative to the reference category of the dependent. Consider the following table (different from the example above) in which high/low Income causes Party = Republican/Democratic affiliation.
```
	
_____________________________________________________________________________							
		Income							
		Low=0	High=1		odds	odds ratio	ln(ratio)		
Party	R = 0	400	500		0.667			-0.405		
	D = 1	600	400		1.25			0.223			
						0.533		-0.629	
             .223 = parameter estimate for party=0
	-.629 = parameter estimate for party=0*income=0	
_____________________________________________________________________________
```
- Positive vs. negative parameter estimates. When we use the exp() function to convert the parameter estimate back into an odds ratio, and the parameter estimate is negative, the odds ratio is the fraction (in this case .53) that the given cell is in relation to the reference cell for that category of the independent variable. That is, significant negative parameter estimates correspond to a decrease in the odds of being in the given category of the dependent variable compared to the the odds of being in that category for people who are in the reference category of the independent factor. For example, in an earlier example where the parameter estimate was -.629, we converted that to an odds ratio of .533 and said therefore the odds of being a Republican if low income was .53 times the odds of being a Republican if high income). If the parameter estimate is positive, the odds ratio will be the multiple that the given cell is in relation to the reference cell on that value of the independent variable. That is, parameter estimates with positive coefficients increase the likelihood of the given response category.
- Reporting odds ratios. The general format of the statement based on the odds ratio is: the odds of someone in the given category of the independent(s) being in the given category of the dependent(s) compared to being in the reference category of the dependent(s) is a multiple (if the parameter estimate is positive) or a fraction (if the parameter estimate is negative) of the odds for a person in the reference category of the independent(s) being in the given category of the dependent(s).
- Logits in logistic regression. Logits are the natural log of the odds ratios. Logits take the form logit(p), where p = the probability that y=1 in binomial logistic regression or y=the highest value in multinomial logistic regression. A positive logit means the independent variable has the effect of increasing the odds that the dependent variable equals a given value (usually 1 for binary dependents, usually the last value for multinomial dependents). A negative logit means the independent variable has the effect of decreasing the odds that the dependent variable equals the given value. For instance, in logistic regression, an odds ratio of 10 means that when the independent variable increases one unit, the odds that the dependent = 1 increase by a factor of 10. (Tip: if you are experimenting using your calculator, note that natural log transformations are very sensitive to rounding off. You cannot go from an odds ratio to a logit, round off to three places, and raise e to the rounded logit and expect to get back the original odds ratio, for instance).
- Multiple independent variables. To estimate the effect of a combination of two independent variables, let L_d be the parameter estimate (lambda) for the dependent category of interest of interest and let L₁ be the estimate for the interaction of the first independent variable's category of interest with this level of the dependent, and let L₂ be a similar estimate for a second independent in the model. To get the odds for the joint effect of the two independent variables each at a given category of interest, calculate exp(L_d+L₁+L₂). The resulting odds ratio is the odds that a person with the given values of the independent variables is in the dependent variable category of interest compared to the same odds for a person with the reference category values on the independent variables. To generalize, the terms within the exp() function are the terms for the model used to generate the estimates.
- Polytomous independent variables. If income had 5 categories rather than two, the interpretation of the parameter estimate for party=0*income=0 would still be the same, but made with respect to people in the highest income category, which would be the reference category. If one wanted to compare income=0 people with income=1 people rather than with the income=5 reference category, one would simply subtract the parameter estimate for income=1 from the parameter estimate for income=0, then convert to an odds ratio using the exp() function, concluding that the odds of being a Republican (party=0) is x times the odds of being a Republican if income = 1, where x is the odds ratio.
- Polytomous dependent variables.. Polytomous (polychotomous) logit models handle the case of a dependent with more than two categories. The dependent is either (1) the odds of being in one dependent category (usually the first) compared to another category as the reference category (usually the last); or (2) in the baseline category approach, the dependent is the odds of being in one category (usually the first) compared to all others as the reference category. The SPSS Logit procedure defaults to having the last dependent category as the reference category.
- Covariates. Continuous variables can be used as independents by entering them into the Cell Covariate(s) box in the main Logit loglinear dialog. Covariates cannot be used to calculate a saturated model. Instead, click the Model button in SPSS and select Custom, then enter a nonsaturated model which includes the covariate(s). SPSS calculates the mean value of the covariate(s) for all observations in the cell formed by the factors (independents) and uses that in calculating estimates. For a dependent with 5 levels, for instance, SPSS will calculate dependent=1*covariate through dependent=4*covariate parameter estimate. The dependent=5*covariate parameter estimate will be 0 since it is the reference category. Note that whereas independents entered as factors have parameter estimates calculated on a case by case basis, independents entered as covariates have parameter estimates calculated based on grouped data (weighted means within cells formed by the factors). There is the possibility that by using means rather than case data, variance will be reduced and the effect sizes reflected in the parameter estimates may be attenuated for covariates compared to factors.
- Confidence limits on parameter estimates: SPSS defaults to 95% confidence limites on estimates, but this can be changed by the researcher under the Options button.
Measures of association. Entropy and concentration measures appear in the SPSS logit loglinear output in the "Analysis of Dispersion" and the "Measures of Association" tables, as illustrated below for the earlier example of Race and Gender predicting Party. The latter table is the one of primary interest. The "Analysis of Dispersion" table contains the data on which the "Measures of Association" table is based. Thus, entropy in the "Measures of Association" table equals model entropy divided by total entropy in the "Analysis of Dispersion" table, and similarly for concentration.

Though sometimes described as being similar to R-square in regression, these effect size coefficients may be small even when the relation between the independent and dependent is strong. Each estimates the percent of the dispersion in the dependent variable which is explained by the model, and both coefficients are usually but not always close to one another.
- Entropy association is model entropy dispersion divided by total (model + residual = total) entropy dispersion. Entropy dispersion is measured by Shannon's entropy coefficient:
  H = -SUM(p_ilogp_i), where p_i is the probability of the dependent in cell i.
- Concentration association is model concentration dispersion divided by total concentration dispersion. Concentration dispersion is measured by Gini's concentration coefficient:
  C = 1 - SUM(p²_i).

Conditional logit models

Conditional logit regression

conditional models in general loglinear modeling

Latent Gold Choice

Stata

SAS

Matched pairs or panel data. The data setup for a conditional logistic regression on matched pairs requires that there be these columns:
1. A variable containing the subject id number, which will be the same for the case and its control
2. A dichotomous dependent variable (response variable), coded 1 for the case and 2 for the control.
3. A copy of the dichotomous dependent variable
4. One or more continuous explanatory variables (covariates) and/or
5. One or more categorical explanatory variables (categorical covariates)
Every subject id will have two data rows: one for the case and one for its control.
In SPSS, select Analyze, Survival, Cox Regression. In the Cox Regression dialog, let the "Time" variable be the dichotomous dependent variable (ex., type - the row is the subject, coded 1, or is the control, coded 2. This means 1 is the "event" condition and 2 is the "censored" condition for each matched pair. Let the "Status" variable be a copy of the dependent variable (ex., type2) and then click the Define Event button and select Single Value and set it to 1. This tells the program that the value 1 corresponds to the event occurring, that is, being the case rather than the control. Enter continuous explanatory variables in the Covariates box (there is an option to do so in blocks). Click the Categorical button and enter any categorical explanatory variables as covariates, choosing the default Indicator contrasts. Back in the main Cox Regression dialog, let the "Strata" variable be a variable giving the subject's id number
Choice models. When conditional logit regression is applied to choices rather than matched pairs, the logic is similar. The dependent is still a dichotomy, but now represents 1=chosen vs. 2=not chosen (censored) for a given alternative. A copy of the dependent is still the Status variable, and it is still set to 1 (assuming 1 = chosen). Each subject id will have two or more rows, for as many rows as there are choices. Continuous or categorical explanatory variables are entered as in matched pairs, and the subject id is entered as the stratification (Strata) variable. Indicator coding is still used.
Output
- The likelihood ratio test appears in the "Omnibus Tests of Model Coefficients" table, in the "Change from previous block" and "Change from previous step" columns. If the explanatory variables have been input using the "Enter" method (the default), these will be the same and a significant likelihood ratio change means the given model is significantly better than the baseline model with only the constant (which is the main effect of the response variable).
- Parameter estimates for each explanatory variable in the "Variables in the Equation" table are associated with the interactions between the dependent (case vs. control, choice vs. non-choice) and the explanatory variables. The researcher concludes that variables whose significance (in the "Sig. column) is equal to or better than .05 are related to the dependent variable.
- Exp(b) in the "Variables in the Equation" table gives the odds ratios associated with each explanatory variable.

Probit

Probit.Probit models, while closely analogous to logit regression models, are not implemented under Analyze, Loglinear, in SPSS. In SPSS the following modules, discussed separately, implement probit:
1. Ordinal regression for response models (in SPSS: Analyze, Regression, Probit)
2. Signal-response models(in SPSS: Analyze, Regression, Ordinal)
3. Ordinal probit regression (in SPSS: Analyze, Generalized Linear Models, Generalized Linear Models)
4. Multilevel probit (in SPSS: Analyze, Mixed, Generalized Linear)
While the links above contain fuller discussion, to summarize briefly here, probit regression is an alternative log-linear approach to handling categorical dependent variables. Its assumptions are consistent with having a dichotomous dependent variable whose distribution is assumed to be a proxy for a true underlying continuous normal distribution. As with logit regression, there are oprobit (ordinal probit for ordered categorical dependents) and mprobit (multinomial probit, for nominal categorical dependents) options. A classic use of probit is to analyze dose-response data in medical studies, where response is 0=dies, 1=lives.
Like logit or logistic regression, in probit regression the researcher focuses on a transformation of the probability that Y, the dependent, equals 1 (ex., response = 1 = lives). Where the logit transformation is the natural log of the odds ratio, the function used in probit is the inverse of the standard normal cumulative distribution function. Where logistic regression is based on the assumption that the categorical dependent reflects an underlying qualitative variable and uses the binomial distribution, probit regression assumes the categorical dependent reflects an underlying quantitative variable and it uses the cumulative normal distribution.
In practical terms, probit models usually come to the same conclusions as logistic regression and have the drawback that probit coefficients are more difficult to interpret (there is no equivalent to logistic regression's odds ratios as effect sizes in probit), hence they are less used, though the choice is largely one of personal preference. Both the cumulative standard normal curve used by probit as a transform and the logistic (log odds) curve used in logistic regression display an S-shaped curve. Though the probit curve is slightly steeper, differences are small. Because of its basis related to the standard normal curve, probit is not recommended when there are many cases in one tail or the other of a distribution. An extended discussion of probit is found in Pampel (2000: 54-68).

Cumulative normal distribution. The S-shaped curve which results when you add up the bell-shaped normal curve, moving from z = -infinity to z=+infinity. The cumulative normal distribution is used by probit and not by logit.
Probit coefficients correspond to the b coefficients in regression or the logit coefficients in logit or logistic regression. All are effect coefficients. Logit and probit analysis generally arrive at the same conclusions for the same data, but the logit and probit coefficients differ in magnitude. Logit coefficients are about 1.8 times the corresponding probit coefficients.
- Meaning. In the case of probit coefficients, the coefficient is how much difference a unit change in the independent makes in terms of the cumulative normal probability of the dependent variable. The effect of a unit change varies by level of the independents. The probit coefficient measures the effect of the independent(s) on the Z scores of the dependent. The probability of the dependent is not a linear function of Z but rather is a cumulative normal function of Z. This means that the effect of a unit change in the independent on the probability of the dependent depends on the level of the independents. Therefore to assess the effect of probit coefficients it is necessary choose some level of the independents as a reference point and in particular the standard reference point is when all independents are at their sample means.
- Use. One substitutes the sample means of the independents into the probit regression equation to get the estimated Z score, then one looks in a table of the standard normal distribution to find the corresponding probability level. This gives us the baseline statement that when all variables are at their sample means, the probability that the dependent will have a value of 1 is that probability. This baseline can then be used to understand the effects of one unit change in an independent, given its probit coefficient value. The answer will be the probability when the independent is at its sample mean plus one unit, minus the baseline probability, with all other independents held constant at their sample means. That is, one takes the baseline equation and simply substitutes the mean of the independent plus 1, then calculates the probability. Then one subtracts the baseline probability from this probability.
- Elasticity. This calculated probability difference is called the elasticity of P(Y) with respect to the independent, when variables are held at their sample means, where Y is the dependent. The elasticity is the effect of a unit increase in the independent variable on the probability that the dependent=1, when all other independents are held constant at their mean values.
Significance testing with chi-square. Note that the chi-square test of goodness of fit cannot be used with PROBIT because it is based on an n by 2 table with one observation per row, which cannot approximate the chi-square distribution even for large samples.
Grouped dose-response models are a type of binomial probit. Such models apply not only to die/live outcomes, but also to a broad range of other types of outcomes, such as not adopted/adopted, not elected/elected, or not hired/hired.
In this grouped dose-response studies, one must input:
1. Response count variable: The dependent is the "response count" variable, which must be coded 0 or 1 (ex., 0 = dies, 1 = lives).
2. Covariate(s): There must be at least one covariate (a continuous independent, not to be represented by a set of dummy variables). The covariate (dosage in medical studies) is conceptualized as the cause.
3. Factor: Optionally, there can be one (and only one) categorical independent. The factor (ex., gender) is conceptualized as a moderating or control variable. Note that if specified as a factor in SPSS, the factor appears in the equation as a set of dummy variables, one for each level of the factor. The intercept is dropped. The coefficient estimates are then the predicted values of each level of the factor with the covariate set to 0.
4. Total observed variable: This is simply a variable which has a value of 1 in all cases. This can be created by the COMPUTE statement. Probit uses this to read the response proportions as 0 out of 1 or 1 out of 1.

Other Logit and Probit Regression Models

Models in Stata. Variants on logit and probit regresion have to do with handling these considerations:
1. Choice-specific vs. case-specific independent variables. Where the dependent is a choice among alternatives, choice-specific independent variables vary both across choices and across cases. Case-specific variables, in contrast, vary only across cases but are uniform within any choice category.
2. Presence of endogenous regressors or not. The model contains endogenous regressors when some of the independents are modeled to be determined by other independent variables.
3. Type of estimation. While maximum likelihood estimation is the usual type, certain variants use other types of estimation as described below.
4. Data level of the dependent. The dependent may be binary, multinomial, or ordered. Variants designed for ordinal dependents use more information than otherwise similar variants which assume only a multinomial level, and hence are considered superior.
5. Fixed versus random effects. Given independent variables may be seen as fixed effects (the usual case) or as random effects (values of the variable are assumed to be a random sample of all possible values, as when the variable "brand" is measured for only 12 of a possible 100 brands).
The Stata package supports a great many variants on logit and probit regression, listed below.
- asclogit. Alternative-specific conditional logit (McFadden's choice) model. This is a variant of conditional logistic regression adapted for data where each data row (observation) is associated with a given choice; where there are multiple observations per choice (these multiple observation records constitute a "case"); and where predictor variables may be identified as choice-specific or case-specific.
- asmprobit. Alternative-specific multinomial probit regression. This is a form of multinomial probit regression, for data where each data row (observation) is associated with a given choice; where there are multiple observations per choice (the "case"); and where predictor variables may be identified as choice-specific or case-specific. A probit model is fitted using maximum simulated likelihood (MSL) estimation, which is more robust than maximum likelihood estimation (MLE, as using in ordinary multinomial logistic regression) against violations of the multinomial logistic assumption that adding or deleting alternatives (choices) does not affect the odds associated with remaining alternatives (this is the IIA assumption - independence of irrelevant alternatives). The asmprobit variant is thus used where there are choice-specific variables and latent variable errors are correlated or heteroscedastic.
- asroprobit. Alternative-specific rank-ordered probit regression. Similar to asmproblit, also using MSL estimation and also allowing case-specific and choice-specific independent variables, but for data where the multiple observations for each case are associated with ranked rather than nominal alternatives.
- biprobit. Bivariate probit regression. For two-equation probit models using maximum likelihood estimation.
- blogit. Logit regression for grouped data, using maximum likelihood estimation.
- bprobit. Probit regression for grouped data. Probit regression for grouped data, using maximum likelihood estimation.
- clogit. Conditional (fixed-effects) logistic regression, used for matched case/control groups. This is the same a fixed-effects logit for panel data. .
- cloglog. Complementary log-log regression models using maximum likelihood estimation.
- exlogistic. Exact logistic regression. Uses exact rather than asymptotic (large sample) maximum likelihood estimation used in ordinary logistic regression. The exlogistic variant produces more accurate estimates for small samples. The exlogistic variant is also an alternative to clogit when the option group(varname) is used.
- glogit. Weighted least-squares logistic regression for grouped data.
- gprobit. Weighted least-squares probit regression for grouped data
- heckprob. Probit model with sample selection. The Stata select() option specifies the sample selection variable(s), at least one of which must not be in the equation for the probit model itself (the outcome equation).
- hetprob. Heteroskedastic probit model, using maximum-likelihood estimation..
- ivprobit. Probit model with endogenous regressors, using maximum likelihood estimation. That is, this variant is for probit models where it is known or assumed that one or more independent variables are determined by other continuous regressor variables.
- logit. Ordinary logit/logistic regression for ungrouped data, using maximum likelihood estimation. The Stata logit command gives logit regression results while the logistic command gives logistic regression results. The parameter estimates are the same and the estimation method (MLE) is the same, but the logistic command outputs odds ratios.
- mlogit. Multinomial logistic regression, for data where the dependent is a categorical variable. Mlogit multinomial logit models use maximum-likelihood estimation for polytomous dependents, and hence mlogit is also known as polytomous logistic regression. Constraints may be defined to perform constrained estimation. The groups formed by the categories of a polytomous dependent are not independent, preventing the researcher from simply doing as many separate logistic regressions as there are categories. Multinomial logit handles non-independence by estimating the models for all outcomes simultaneously except, as in the use of dummy variables in regression, one category is "left out" to serve as a baseline. Since effects must sum to zero, the model for the reference group can be reproduced from other parameters in the output. (Note: some authors use the term "multinomial logit" to refer to what is conditional logit, discussed above.)
- mprobit. Multinomial probit regression using maximum likelihood estimation.
- nlogit. Nested logit regression, using full information maximum likelihood (FIML) estimation. FIML estimation is appropriate where, as with nested data, one is apt to violate the assumption of independently distributed errors. The nlogit variant is also appropriate for conditional and multinomial logit models when similar alternatives are clustered, causing violation of the assumption of independence of irrelevant alternatives (that adding or removing alternatives does not affect odds of the remaining alternatives).
- ologit. Ordered logistic regression, used when the dependent variable is ordinal rather than binary or multinomial. Ordinal log-linear analysis uses the same goodness-of-fit tests and has the same interpretation of coefficients as nominal log-linear analysis. Ordinal logit or probit models estimate the the probability that a case will fall in the �next highest� category of the dependent variable. A chi-square test is applied to test the assumption that the effects of a unit of the independent variable on the probability of the next highest category of the dependent variable are the same for all levels of of the dependent (this is the proportionality assumption in ologit and oprobit). Note that ordinal logit or probit estimate k-1 intercept terms, where k = the number of categories of the dependent variable. These intercepts can be interpreted as indexing the distance between sequential points on the ordinal scale of the dependent variable.
  - Likelihood ratio chi-square test. The is the overall test of the fitted ologit model. Stata outputs "LR chi2(3)" (the parenthetical term is degrees of freedom), whose corresponding significance level is reported as "Prob > chi2". A finding of non-significance indicates that at least one of the terms (at least one of the ologit coefficients) in the model differs significantly from zero.
  - Psuedo-R². An effect size measure for the overall model, reported by Stata as "Pseudo R2," this is McFadden's pseudo R-square. It is not equivalent to R² in OLS regression and should not be interpreted as percent of variance explained. in See further discussion in the section on logistic regression.
  - Ordered logit coefficients. Corresponding to b coefficients in OLS regression, these are the ordinal logits, also called ordered log-odds regression coefficients. Reported by Stata as parameter estimates in the "Coef." column of output, an increase of 1 unit in the predictor variable is associated with a b change in the dependent variable (as measured in the ordered log-odds scale), controlling for other variables in the model. That is, the ordered log-odds of being in a higher level of the ordinal dependent are increased (or decreased if the sign is negative) by the amount of a predictor's ordered logit when that predictor increases 1 unit, controlling for other variables.
  - Ancillary parameters, a.k.a. cutpoints. Reported by Stata as _cut1, _cut2, etc., there will be one fewer cutpoints than levels of the ordinal dependent. These cutpoints are used when classifying subjects into levels of the ordinal dependent variable. Note that interpretation of the ordered logit coefficients does not depend on the cutpoints in terms of significance. However, the cutpoints define the ordered log-odds scale of the dependent and affect the cumulative probabilities of classification of cases on levels of the dependent. Differences in cutpoints may account for differences in findings when applying ologit in Stata versus ordinal regression in SAS or SPSS.
  - Standard error of the ordered logit. Reported by Stata As "Std. Err.", this is the standard error of the ordered logit coefficient. A z test is applied and the corresponding probability level (labeled "P>|z|" by Stata) derived. Coefficients with a p level at or below the alpha significance level (typically .05) may be considered significantly different from zero. Confidence limits are also output by Stata.
  - Odds ratio. Proportional odds ratios may be output in Stata by adding the ", or" parameter after the variable list in the ologit command line. Output will then list the "Odds Ratio" for each predictor variable instead of the ordered logit coefficient. Standard errors, probability levels, and confidence intervals will also be output. The probability levels will be identical to those for the corresponding ordered logit coefficients. Odds ratios are the natural log e raised to the power of the b coefficient.
    Odds ratios compare the odds of being at a given level of the ordinal dependent or higher, compared to being lower. For instance, consider an odds ratio of 2.0 for a dependent with four levels. For a 1 unit increase in the predictor, the odds of being in 4th level compared to levels 1-3 is 2:1, holding constant other predictors. Likewise, the odds of being in the 3rd level or higher compared to levels 1-2 is also 2:1.
  - Assumption of parallel regressions. As in ordinal regression in SPSS, ologit assumes that the regression slopes are the same for each level of the ordinal dependent. This is a critical assumption and must be tested by the Brant test of parallel regression lines, using the brant command in Stata. When data fail the Brant test, the researcher may wish to consider modeling the data in mlogit (see above). When the assumption is met, ologit is more powerful. Alternatively, it may be possible to alter the number of levels of the dependent to achieve parallelism of regressions.
- oprobit. Ordered probit regression, used when the dependent variable is ordinal rather than binary or multinomial.
- probit. Ordinary probit regression, using maximum likelihood estimation. Note that whereas probit reports parameter estimates, the alternative dprobit command reports marginal effects in terms of probabilities.
- rologit. Rank-ordered logistic regression, using maximum likelihood estimation. Also called exploded logit regression or choice-based conjoint analysis. The dependent variable reflects rankings among choices.
- scobit. Skewed logistic regression. This variant fits a logistic model using maximum likelihood estimation, for a skewed dependent variable.
- slogit. Stereotype logistic regression, using maximum likelihood estimation. This variant is used for categorical dependents when the researcher is unsure whether the classes are unranked (and one would use multinomial logistic regression) or ranked (and one would use ordinal logistic regression). For instance, the researcher may suspect some alternatives are similar while others are clearly ranked.
- xtcloglog. Random-effects and population-averaged cloglog models, used when the researcher wants a complementary log-log model but some independent variables are assumed to be random rather than fixed effects.
- xtlogit. For conditional fixed-effects, random-effects, and population-averaged logit models.
- xtprobit. For random-effects and population-averaged probit models.

Tobit

Developed by Tobin in the 1950s, tobit is a nonparametric alternative to OLS regression or ANOVA, often used when variables have extreme skews and thus do not meet parametric assumptions. Tobin, for instance, used his procedure in consumer research where for certain items (variables) the bulk of consumers made no purchase. Substance abuse and crime pattern research often covers equally skewed variables. Tobit uses maximum likelihood estimation (MLE) to yield coefficient estimates of the probability of being above the modal value (usually 0 = no purchase, no substance abuse, etc.). SPSS does not support tobit (apart from manual computation in the SPSS MATRIX module). However, LimDep, an econometrics package does. SPSS does support logistic regression, also a MLE approach, which may achieve some of the same purposes. That is, one may regress a given outcome (ex., 1 = purchase, 1 = substance abuse, etc.) for a set of predictors using logistic regression to assess the effect of each predictor and the significance of their collective effect on the outcome.

Poisson regression

Poisson regression is a form of analysis common in event history analysis and other research involving rare events where assumptions of a normally distributed dependent do not apply. It is also implemented in generalized linear modeling (GZLM), discussed separately. In the context of loglinear analysis in SPSS, selecting the Poisson distribution is an option under Analyze, Loglinear, General. In fact, it is the default assumption under HILOG and GENLOG.
Rate variables as dependents in loglinear Poisson regression. In a simple loglinear model involving counts as a dependent variable (ex., suicides) and various nominal variables as predictors (ex., region, religion), data are weighted by the count variable (Data, Weight Cases, in SPSS). What differentiates a loglinear model as Poisson regression is that the researcher is interested in predicting a rate, not a count (ex., rate of suicides, not count of suicides). In a Poisson regression involving rates rather than counts as a dependent variable, an additional variable representing the denominator must be added as a Cell Structure variable in the Analyze, Loglinear, General dialog box. For instance, for a model of suicides in various sampled cities, the additional variable might be city population. Thus the normal data setup for this example would be region category as the first column, religion category as the second column, number of suicides (for that region-religion combination) as the third column, and population size (for that region-religion combination). Then in Data, Weight Cases, the count variable number of suicides would be the weighting variable; and region and religion would be entered in the loglinear model as factors and population would be the Cell structure variable in the Analyze, Loglinear, General dialog.
Predictor variables in loglinear Poisson regression. Continous predictor variables may be added as covariates in either count or rate Poisson regression models. If there are only continuous predictors and no other factors, a subject id variable is added with values which are unique to each case. In the Analyze, Loglinear, General dialog box, the continuous variables are specified as Cell covariates and the subject id variable is specified as a Factor. Any 0 marginals will have to be recoded to a very small positive number (ex., 1E-12) or GENLOG will refuse to compute maximum likelihood estimates.
Discussion. Like other forms of loglinear analysis, Poisson regression is predicting the count or rate associated with each cell in the table formed by the factors (region and religion in this example). For the example of predicting suicide rate for each table cell, SPSS will produce a table of "Parameter Estimates," with one estimate for each cell, and it will produce a constant. These are the mu and lambda coefficients discussed above in the section on general loglinear modeling. The last category of region and of religion will be the reference category (hence it would have been best if the researcher had set up the data with the desired reference category being the highest-coded). If the parameters are negative, then that cell has a fraction of the suicide rate of the reference category. For instance, if the parameter for region=1 is -2.24, since exp(-2.24)=.106, region 1 has 10.6% the suicide rate of the reference region. The greater in a negative direction the estimate, the smaller the fraction. A positive parameter would have meant the region had a risk which was a multiple of the reference group. The greater in a positive direction the parameter, the greater the multiple. By way of comparison, logit coefficients tend to be about 1.7 times larger than probit coefficients, unless an independent variable has a large standard error.
If you ask for Predictions under the Save button, SPSS will add a column of predicted counts (labeled PRE_1 on the first run) to the working dataset. In the example, this would be the predicted count of suicides for the given cell. The predicted rate can be calculated based on the Poisson regression model:

Assumptions

Not assumed: like logistic regression, loglinear and logit methods have relaxed data assumptions compared to OLS regression. There is no assumption that the dependents and independents be related linearly. There is no assumption of homoscedasticity. The dependent need not be normally distributed. The independents need not be interval in level of measurement, nor need they be unbounded.
Well-populated tables. Log-linear analysis focuses on categorical variables, whose intersections are tables. Sparse cells in the table may well render log-linear analysis unreliable. If many cells have small expected values, both likelihood ratio and Pearson chi-square may mis-estimate the true significance. A rule of thumb is that expected frequencies should be >= 1 for all cells and at least 80% of cells should have expected frequencies > 5. Note this assumption pertains to expected frequencies, not observed frequencies (even observed frequencies of 0). When cell frequencies are small, SPSS's default addition of .5 to all cells (the delta factor) can appreciably alter results obtained. It also reduces power. Expected frequencies are calculated as the product of row and column marginals divided by n, the sample size. As this assumption is violated, the power of the log-linear procedure diminishes (more likelihood of Type II errors, thinking there is no relationship when there is a relationship). Signficance tests, which deal with Type I error, are usually not affected by violation of this assumption. Collapsing rows and columns to create better-populated tables also decreases power.
- Small models with few variables. A corollary is that large models with many variables and variables with many levels will usually fail to have well-populated tables and thus will not be suitable for log-linear analysis. This is a limitation with all categorical data analysis: results pertain only to cells with adequate expected frequency.
- Adequate sample size. Well-populated tables also require adequate sample size. Moreover, both logit and probit models utilize maximum likelihood estimation methods, which require a larger sample size than the corresponding OLS regression methods. A rule of thumb is that sample size should be a bare minimum of five times the number of cells in factor space (multiply the number of levels of each factor). Inadequate sample size will lead to too many cells with zero count and log-linear analysis may fail to converge on a solution.
- Adding .5 to each cell is a default strategy taken by SPSS and others, to avoid having 0-count cells. However, this strategy improves Type I error only minimally while reducing power (that is, Type II errors are increased). The .5 default value is "Delta" under the Options button in SPSS, where the researcher may reset it to 0. Resetting to Delta = 0 is widely recommended.
Small residuals (no large outlier cells). Outlier frequences are observed cell frequencies greatly different from expected frequencies. Outlier cells are flagged in log-linear residual analysis by having standardized residuals > 1.96, for the alpha=.05 significance level. Unusually large outliers may mean that no model can achieve goodness of fit. Sometimes adding variables or collapsing rows or columns can improve fit.
Normally distributed residuals. As detected in Q-Q plots discussed above, non-normal distribution of residuals indicates the loglinear model is under- and/or over-estimating certain ranges of cases. Residual analysis helps the researcher identify outlier cells in the table being analyzed, and this in turn may help the researcher respecify the loglinear model.
Categorical data. If continuous data are categorized to make them "work" in log-linear analysis, information is lost and correlations may be attenuated, yielding misleading results.
Evenly distributed categories. Logit loglinear analysis assumes evenly distributed categories. If the categorical dependent variable is assumed to be a proxy for an underlying latent variable which is normally disributed, probit may be a more appropriate choice. In general, however, logit and probit yield very similar substantive results.
Independence. Observations are assumed to be independent. Knowing case n does not predict case (n+1). Thus, log-linear analysis is not appropriate for correlated observations as found in time series, before-after, panel study, and other within-groups, and other repeated measures designs. However, conditional logit models exist which will handle correlated data.
Data distribution assumptions. The researcher has the option in SPSS general log-linear modeling to assume a Poisson distribution of cell frequencies, or a multinomial distribution. The usual choice is Poisson distribution, which is used to predict rare events and cell frequencies when neither sample size nor row and/or column totals are predetermined in advance. Multinomial distribution is assumed when sample size and row and/or column totals are predetermined (sampling has been stratified, for instance, in a race by religion table, the numbers of white Protestants, black Catholics, etc., are predetermined). Thus there are four possibilities:
1. Sample size is not fixed, nor are row/column marginals. Use the Poisson distribution.
2. Sample size is fixed, but cell frequencies are independent. This is the simple multinomial distribution but if analysis is not conditional on total sample size (the usual case), the Poisson distribution is selected.
3. Sample size and row or column marginals are fixed in advance. This is a product multinomial distribution, and the researcher should select multinomial distribution.
4. Sample size and both row and column marginals are fixed in advance. This is a hypergeometric distribution, and the researcher should select multinomial distribution.
Logit loglinear models assume a multinomial distribution of counts within each combination of categories of independent variables.
Appropriate dispersion. Expected and observed variances of the dependent should be checked for under- and over-dispersion, as in logistic regression. However, in loglinear analysis the expected variance(y) equals simply the mean of y. As in logistic regression, if there is great under- or overdispersion, then there is a problem with model specification or other severe design problems. If the dispersion is moderate, one corrects by using adjusted standard error = SE * SQRT(D/df), where D is scaled deviance, which for loglinear analysis equals G² and df = the number of terms in the model not counting the constant.
Absence of endogenous regressors. The ordinary probit model assumes that all independents are exogenous variables which directly cause the dependent variable. If the researcher wishes to assume a model in which some of the independent variables are seen as being determined by other independent variables, a variant is available called ivprobit, discussed above.
See logistic regression section for other assumptions.

Frequently Asked Questions

Why not just use regression with dichotomous dependents?
Why not just use crosstabulation and ordinal measures of association rather than ordinal log-linear analysis?
What computer packages implement log-linear analysis?
What are second-order and partial odds ratios?
What are structural zeros and sampling zeros in the SPSS "Data Information" table?
Since logit and probit generally lead to the same statistical conclusions, when is one better than the other?
Do I really need to do multinomial logit (multinomial logistic regression) or multinomial probit? Could I just apply M different logit or probit models for a variable with M levels?
What if my variables are multiple-response type?
Explain "partial odds.".
Explain coding in saturated vs. nonsaturated models.
What is log-linear analysis with latent variables?

Bibliography

Agresti, A. (1996). An introduction to categorical data analysis. NY: John Wiley.
Allison, Paul D. and Nicholas Christakis (1994). Logit models for sets of ranked items. Pp. 199-228 in Peter V. Marsden, ed., Sociological Methodology 1994. Oxford: Basil Blackwell. Covers conditional logit models.
Anderson, E. B. (1996), Introduction to the statistical analysis of categorical data. NY: Springer-Verlag.
Borooah, Vani Kant (2002). Logit and probit. Thousand Oaks, CA: Sage Publications. Compares odds ratios vs. risk ratios, logit vs. probit, multinomial logit, step-by-step approach to procedures.
Christensen, R. (1997). Log-linear models and logistic regression. NY: Springer-Verlag.
DeCarlo, Thomas E., Laczniak, R. N., Azevedo, K. A., and Ramaswami, S. N. (2000). On the log-linear analysis of multiple response data. Marketting Letters 11(4): 349-361.
Friendly, M. (2000). Visualizing categorical data. Cary, NC: SAS Press. Focus on expressing loglinear results in plots.
Gilbert, N. (1993). Analyzing tabular data: Loglinear and logistic models for social researchers. London: UCL Press. An introductory text.
Hagenaars, Jacques A. (1993). Loglinear models with latent variables. Quantitative Applications in the Social Sciences Series # 94. Thousand Oaks, CA: Sage Publications. This is discussed in the "Frequently Asked Questions" section above.
Hardin, J. W., & Hilbe, J. M. (2007). Generalized linear models and extensions (2nd ed.). College Station, TX: StataCorp LP.
Hilbe, J. M. (2007). Negative binomial regression. New York: Cambridge University Press.
Knoke, D. & Burke, P. J. (1980), Log-linear models. Newbury Park, CA: Sage Publications.
Liao, Tim Futing (1994). Interpreting probability models: Logit, probit, and other generalized linear models. Thousand Oaks, CA: Sage Publications. Presents the principles of binary, sequential, ordinal, multinomial, and conditional logit models. Also covers probit models and Poisson regression. Does not cover SPSS or SAS input/output issues.
Pampel, Fred C. (2000). Logistic regression: A primer. Sage Quantitative Applications in the Social Sciences Series #132. Thousand Oaks, CA: Sage Publications. Pp. 54-68 provide a discussion of probit.
Rudas, Tamás (1998). Odds ratios in the analysis of contingency tables. Thousand, Oaks, CA: Sage Publications. Quantitative Applications in the Social Sciences Series No. 119. Good basic explanation of odds ratios, many examples, also focuses on use of conditional odds ratios in analyzing crosstabulated data.
Vermunt, Jeroen K. (1997). Log-linear models for event histories. Thousand Oaks, CA: Sage Publications. Focuses on log-linear models using a Poisson transformation for Poisson regression rather than the log of the odds ratio (logit or logistic regression) or the cumulative normal curve (probit regression). Liao (1994) also has a section on Poisson regression. See also event analysis.
Wickens, T. D. (1989). Multiway contingency table analysis for the social sciences. NY: Springer Verlag.

Copyright 1998, 2008, 2009, 2010, 2011, 2012 by G. David Garson.
Do not post on other servers, even for educational use.
Last update 3/5/2012.