![[Home]](redbar2.jpg){kind=small}
![[Syllabus]](nav1.jpg){kind=small}
![[Statnotes]](nav2.jpg){kind=small}
![[Links]](nav3.jpg){kind=small}
![[Lab]](nav4.jpg){kind=small}
![[Instructor]](nav5.jpg){kind=small}
|
|
|
OverviewHLM software is perhaps the leading statistical package for hierarchical linear modeling. While many of the models discussed in this section can be accomplished with SPSS mixed models or SAS PROC MIXED, many researchers find HLM to be a more versatile and full-featured environment for hierarchical linear modeling. Only some of its many features are discussed here. Two major advantages of HLM software over SPSS for hierarchical linear modeling are (1) the ability to specify any of several distributions for the dependent variable rather than assume a continuous normal distribution (hence, for instance, the ability to conduct logistic hierarchical linear regression); and (2) the ability to handle heterogenous hierarchical linear models (where the dependent is thought to have different variances for different levels of some grouping variable such as gender or race, for instance). HLM software. Scientific Software International (SSI) distributes HLM. A free student edition is available at http://www.ssicentral.com/hlm/student.html. The student edition is full-featured, including examples, but is limited in the size and complexity of models (though it will work all example files). |
|
After selecting the File, Make new MDM file, Stat package input option illustrated above, the researcher chooses the HLM model type wanted in the "Select MDM type" dialog illustrated below. For the data above, a simple two-level hierarchical linear model is appropriate:
The next dialog box, illustrated below, lets the researcher browse and select the level 1 and level 2 data files; name the .mdm file to be generated (here hsb.mdm); open/save/edit .mdm template files (.mdmt files); and handle missing data. There is a "Choose Variables" button for level 1 and level 2, and the researcher checks the first column checkbox for the id (link) variable(s) and checks the second column checkboxes for the variables at each level to be used and included in the .mdm file.
To complete the process, the researcher clicks the "Make MDM" button and then clicks the "Check Stats" button to verify the results, as shown below. The "Check Stats" output is descriptive statistics on the variables brought into the new .MDM file. Warning: If the sample size seems low, it may be the researcher has not sorted the Level 1 file to assure individual rows for the same level 2 id (school id in this example) are adjacent.
At this point, the researcher will have saved three files to the disk: the newly created HLM-compatible data file, HSB.MDM in this example; the default template creatmdm.mdmt; and the output file above, HLM2MDM.STS (if desired, use File, Save As, to save output under a different name as this default file may get re-used with new content).
Click "Done" on the "MAKE MDM - HLM2" dialog to exit to the WHLM model construction screen discussed below.
Warning. Prior to running multilevel models, the researcher should rule out multicollinearity among the level 2 (or higher) predictors. Also, as in all models, the researcher should be aware that changes in what variables and effects are specified may well change coefficient estimates substantially.
Clicking the "Basic Settings" menu choice in the WHLM dialog allows the research to specify the distribution of the dependent variable, including normal, Bernoulli, Poisson, Multinomial, and Ordinal distributions, as illustrated below.
Hierarchical logistic models. A normal distribution is the default, used in the examples below. Selecting Bernoulli for a binary outcome variable applies a logistic link function and, as in logistic regression, making interpretations in terms of the log odds of the outcome rather than in terms of the raw outcome itself. The intercept ceases to be the mean value of the outcome but instead is the mean log odds of the outcom across level 2 units in a 2-level model. Also, there will be no level 1 variance component since the outcome is binary, and this means there is no simple intraclass correlation coefficient (discussed below for normally distributed outcome variables).
Other models. In the dialog below, the researcher may also select "Multinomial" as the distribution of the outcome variable, causing a multinomial logit link to be employed, akin to multinomial logistic regression. There is also an option for hierarchical ordinal regression models. Hierarchical Poisson models employ a Poisson log link and require an exposure variable (time, for ex.).
Click "Run analysis" from the menu to generate output for the intercept-only model, shown below in excerpted form. Click File, View output to view. Comments in italics are not part of the output.
Program: HLM 6 Hierarchical Linear and Nonlinear Modeling ... ------------------------------------------------------------------------------- Module: HLM2S.EXE (6.06.2857.2) ... SPECIFICATIONS FOR THIS HLM2 RUN The data source for this run = hsb.mdm ... The outcome variable is MATHACH ... Summary of the model specified (in equation format) --------------------------------------------------- Level-1 Model Y = B0 + R Level-2 Model B0 = G00 + U0Model specification. B0 is the level 1 intercept. Y is the dependent, MATHACH math achievement scores in this example. Math achievement in this null model is a function of the intercept plus a residual (error) term. The level 1 intercept, B0, in turn is a function of the grand mean across level 2 units (schools) plus a random error term, signifying the intercept is modeled as a random effect. The id variable contains the id value for any given level 2 unit (school).
Iterations stopped due to small change in likelihood function ******* ITERATION 4 ******* Sigma_squared = 39.14831 Tau INTRCPT1,B0 8.61431 ...Sigma-squared is the level 1 variance in the intercept, reflecting mean MATHACH of students within schools. It corresponds to the "Residual" estimate in the SPSS "Estimates of Covariance Parameters" table.
Tau is the level 2 variance between schools of the intercept. It corresponds to the "Intercept[subject=id] Variance" estimate in the SPSS "Estimates of Covariance Parameters" table.
Intraclass correlation (ICC). In a variance components model (which assumes independent random effects with uncorrelated error, hence zero covariance of error terms for random effect slopes and intercepts), the total variance is sigma-squared plus tau: 39.15 + 8.61 = 47.76. Also, as here, if there is only one random effect, there is no term for covariance of error terms. As will be discussed below, however, when there are two or more random effects, and when a variance components model does not apply, the computation of total variance must include covariance of error terms.
The intraclass correlation (ICC) is the percent of variance in MATHACH attributable to the between schools effect, calculated in this example as = 8.61/ 47.76 = .18. The proportion of MATHACH attributable to within-schools effects in this null model is = 1 - .18 = .82. This is parallel to the variance components calculation described with respect to VC models in the section on SPSS linear mixed models.
ICC varies from +1.0 when group means differ but within any group there is no variation, to -1/(n-1) when group means are all the same but within-group variation is very large. ICC is sometimes used to assess the utility of applying a hierarchical linear model. At the extreme, when ICC approaches 0 or is negative, hierarchical modeling is not appropriate.
---------------------------------------------------- Random level-1 coefficient Reliability estimate ---------------------------------------------------- INTRCPT1, B0 0.901 ----------------------------------------------------Reliability. HLM treats the intercept as a random effect. A regression was run on each of the 160 level 2 units (schools). The reliability of the intercepts was .901. If the reliability were 1.0, there would be no difference between estimates of slopes and intercepts in HLM compared to OLS regression. In reality, reliability is always less than 1.0. The lower the reliability, the more HLM and OLS estimates will diverge because when calculating the intercept (or in later models, the slopes of the predictors) HLM weights the 160 coefficients such that intercepts or slopes for schools with greater reliability count more than those for schools with lower reliability.
The value of the likelihood function at iteration 4 = -2.355840E+004
The outcome variable is MATHACH
Final estimation of fixed effects:
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, B0
INTRCPT2, G00 12.636972 0.244412 51.704 159 0.000
----------------------------------------------------------------------------
Confidence limits. The Level 1 intercept is 12.64, representing the mean math achievement score among schools. The 95% confidence limits on this value of B0 may be calculated as follows: Since tau is the variance of the intercept, its square root is the standard deviation: s.d. = SQRT(tau) = SQRT(8.61431) = 2.9350. The 95% level corresponds to plus or minus 1.96 standard deviations, so 95% of the 160 school regressions may be expected to have an intercept (mean score) between a high of 12.64 + 1.96*2.935 = 18.39 and a low of 12.64 - 1.96*2.935 = 6.88.
Final estimation of fixed effects
(with robust standard errors)
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, B0
INTRCPT2, G00 12.636972 0.243628 51.870 159 0.000
----------------------------------------------------------------------------
Distribution of the dependent. Robust standard errors are advisable when there is misspecification of the distribution of the dependent variable. Therefore significant differences between the ordinary and robust estimates of the standard error may flag a problem with the distribution specified by the researcher (normal is default). This does not appear to be a problem for this example. This specification may be changed in the "Basic Settings" dialog as discussed and illustrated above.
Final estimation of variance components:
-----------------------------------------------------------------------------
Random Effect Standard Variance df Chi-square P-value
Deviation Component
-----------------------------------------------------------------------------
INTRCPT1, U0 2.93501 8.61431 159 1660.23259 0.000
level-1, R 6.25686 39.14831
-----------------------------------------------------------------------------
Model significance. The variance components are the tau and sigma-squared values discussed above. The standard deviations are their square roots. Degrees of freedom are (n - 1), where n is the 160 schools in the sample. The model as a whole is significant, meaning the intercept is significantly different from 0 (also reflected in the fact that 0 is not within the confidence limits calculated above).
Statistics for current covariance components model -------------------------------------------------- Deviance = 47116.793477 Number of estimated parameters = 2 -----------------------------------------------------------------------------Likelihood ratio test. Deviance is the -2 log likelihood (-2LL) coefficient. The difference between the deviance for the intercept only model and a model with level 1 and/or level 2 predictors is used in the likelihood ratio test to assess the effects of adding predictors to the model. The more adding a predictor lowers deviance, the more effect it has on predicting MATHACH. Likelihood ratio tests are discussed in the sections on logistic regression and on structural equation modeling.
To illustrate, in HLM the level 1 predictor, student SES, is added to the level 1 equation as illustrated below. As SES is already standardized, it is not added under the centering option which HLM provides. The intercept and slope are modeled as random coefficients.
Note: HLM gives the option of toggling the level 2 error terms on or off - here we right-click on the error terms to make sure they are "on" rather than in the greyed-out "off" position. In the figure below, the U1 error term for the level 2 estimator of the level 1 slope of SES (B1) is turned on. U1 is the level 2 random effect for the level 1 B1 slope, making this a random coefficients model. If U1 were toggled off, only the B0 intercept would be estimated by a level 2 random effect (U0), making this a random intercepts model rather than a random coefficients model.
Click "Run analysis" from the menu to generate output for the random coefficients model, shown below in excerpted form. Click File, View output to view. Comments in italics are not part of the output.
Program: HLM 6 Hierarchical Linear and Nonlinear
Module: HLM2S.EXE (6.06.2857.2)
The data source for this run = hsb.mdm
The outcome variable is MATHACH
The model specified for the fixed effects was:
----------------------------------------------------
Level-1 Level-2
Coefficients Predictors
---------------------- ---------------
INTRCPT1, B0 INTRCPT2, G00
SES slope, B1 INTRCPT2, G10
Summary of the model specified (in equation format)
---------------------------------------------------
Level-1 Model
Y = B0 + B1*(SES) + R
Level-2 Model
B0 = G00 + U0
B1 = G10 + U1
Model. MATHACH (Y = math achievement score) is a function of the intercept (B0) plus the slope times the predictor (B1*SES) plus a residual error term. The intercept (B0) is a function of the grand mean of MATHACH across level 2 units (schools) plus a random error term. The slope (B1) is a function of the grand mean of SES across schools plus a random error term. Again, clicking the "Mixed" button algebraically substitutes the level 2 equalities into the level 1 equation to create a single mixed equation for the model, shown in the figure above.
Iterations stopped due to small change in likelihood function
******* ITERATION 21 *******
Sigma_squared = 36.82835
Tau
INTRCPT1,B0 4.82978 -0.15399
SES,B1 -0.15399 0.41828
...
Sigma-squared is the within-schools variance in MATHACH after SES is controlled, here 36.83. For the same data and model, this corresponds to the "Residual" estimate in the "Estimates of Covariance Parameters" table in SPSS output.
Tau. The tau values now appear in a matrix and are level 2 interaction effects with the level 1 outcome variable. The intercept value in the upper left of the matrix (here 4.83) is the between-schools variance estimate for the intercept, assuming no level 2 predictors. It corresponds to the "Intercept[subject=id] Variance" estimate in the "Estimates of Covariance Parameters" table of SPSS output for the same model and data.
The tau value in the lower right (here .42) is the between-schools variance estimate for the slopes of SES, assuming no level 2 predictors. Note "slopes" is plural because separate regressions are computed for each level 2 unit, school. This tau value corresponds to the "ses[subject=id] Variance" estimate in the "Estimates of Covariance Parameters" table of SPSS output for the same model and data.
The off-diagonal elements in the tau matrix represent the covariance of the error terms for the level 2 estimates of the level 1 slopes and intercepts (-.15). That the covariance value is negative means that as mean math achievement (intercepts) go down among the 160 schools, the relationship of individual-level SES and math achievement (slopes) becomes more important (stronger, goes up).
Intraclass correlation (ICC). With two random effects (here the intercept and slope of centered ses), the total of variance components is the within-group variance (sigma-squared = 36.82835; this is the level 1 variance term) plus the level 2 between-group variance terms: 4.82978 (the level 2 intercept variance estimate) + 0.41828 (the level 2 slope variance estimate) plus twice the covariance: 2*-0.15399. For these data, the total variance sums to 41.76843. For a variance components model, constrained to have zero covariance of the random effect error terms, there would be no 2*cov term in the ICC formula. For other models the covariance must be taken into account because the variance components are no longer additive in a simple way. The ICC is the between-groups effect on the intercept of the outcome variable (4.82978) divided by total variance (41.76843) = .12. We may say that 12% of the variance in math achievement is attributable to the between-groups effect.
Residual variation. Sigma-squared and tau coefficients are partial coefficients and must be interpreted differently than in the intercept-only model above. That is, they reflect residual variation after controlling for predictors in the model. Likewise, the intraclass correlation (ICC) based on these coefficients also becomes a partial coefficient, also reflecting R2 after controls are applied.
R2 estimate. Sigma-squared, the variance of MATHACH within schools after SES is controlled, is 36.83. In the intercept-only model it was 39.15. The difference is 2.32. Adding individual level (level 1) SES to the model thus reduces within schools variance of MATHACH by 2.32/39.15 = .06. This value is an estimate of R2 for the random coefficients model with SES (a level 1 variable) as the only predictor.
Models without intercepts. Though not the case here, it may be noted that it is possible to delete the level 1 intercept (B0) from the model. This is done when there is a complete set of dummies as level 1 predictors. For instance, if there is a "MALE" variable coded 0,1 with 1=Male, and there is a "FEMALE" variable coded 0,1 with 1=Female, the model would be be overdetermined (lack positive degrees of freedom needed for solution). To prevent this, the B0 term may be deleted. In the main HLM interface, simply select the Level 1 list of variables, highlight INTRCPT1, then delete it.
Reliability. The intercept is much more reliable than the slope of SES, reflecting the fact that SES is not a powerful predictor.
----------------------------------------------------
Random level-1 coefficient Reliability estimate
----------------------------------------------------
INTRCPT1, B0 0.797
SES, B1 0.179
----------------------------------------------------
...
Final estimation of fixed effects:
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, B0
INTRCPT2, G00 12.664935 0.189874 66.702 159 0.000
For SES slope, B1
INTRCPT2, G10 2.393878 0.118278 20.240 159 0.000
----------------------------------------------------------------------------
Final estimation of fixed effects
(with robust standard errors)
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, B0
INTRCPT2, G00 12.664935 0.189251 66.921 159 0.000
For SES slope, B1
INTRCPT2, G10 2.393878 0.117697 20.339 159 0.000
----------------------------------------------------------------------------
Average intercepts and slopes. The fixed effects table above shows the average intercept and slope across the 160 schools. Here, the predictor is SES and thus the coefficient is a slope. Had the predictor been categorical, the coefficient could be interpreted as differences between means. Either way this a partial coefficient, controlling for variation in students within schools. That standard errors above are very similar whether calculated by the ordinary formula or the "robust" method, indicating that the default assumption that the dependent variable is distributed normally is acceptable. By either calculation, both the intercept and the slope of SES are found to be significantly different from 0 in this model.
Final estimation of variance components:
-----------------------------------------------------------------------------
Random Effect Standard Variance df Chi-square P-value
Deviation Component
-----------------------------------------------------------------------------
INTRCPT1, U0 2.19768 4.82978 159 905.26472 0.000
SES slope, U1 0.64675 0.41828 159 216.21178 0.002
level-1, R 6.06864 36.82835
-----------------------------------------------------------------------------
Confidence limits. In the fixed effects table further above, the intercept (B0) was 12.66. Its confidence limits will be plus or minus 1.96 standard deviations of that: 12.66 ± 1.96*2.20 = high confidence limit or 16.97 and low confidence limit of 8.35. Similarly, the slope of SES in the fixed effects table was 2.39, so its confidence limits are 2.39 ± 1.96*.65 = high of 3.66, low of 1.12. That 0 is not in either confidence range shows that both intercept and slope are significantly different from 0 by the usual 95% confidence criterion, which corresponds to 1.96 standard deviations. More specifically, we reject the null hypotheses that slopes and intercepts have no difference between schools.
Statistics for current covariance components model -------------------------------------------------- Deviance = 46638.560929 Number of estimated parameters = 4 -----------------------------------------------------------------------------Deviance. Above we see deviance has dropped somewhat, from 47116.79 in the intercept-only model to 46638.56 in the present random coefficients model with SES as a level 1 predictor. This is a difference of 478.23. The intercept-only model had 2 parameters, whereas the present model has 4 - a difference of 2. The critical value of chi-square for 2 degrees of freedom is only 13.82 (read from a chi-square table), far lower than the difference, so we may say the difference between models is significant at ≤ .001. Deviance corresponds to the "-2 Restricted Log Likelihood" estimate in the "Information Criteria" table of SPSS output for the same model and data.
Having chosen the hypothesis testing option causes the likelihood rato test (corresponding to the p-value of the chi-square statistic in the output below) to be printed in the Deviance section of HLM output (bottom of the output). Again, the likelihood ratio test shows the random coefficients model with SES as a predictor to be different from the intercept-only model at a significance level ≤ .001:
Statistics for current covariance components model -------------------------------------------------- Deviance = 46638.560929 Number of estimated parameters = 4 Variance-Covariance components test ----------------------------------- Chi-square statistic = 478.22907 Number of degrees of freedom = 2 P-value = 0.000
Click "Run analysis" from the menu to generate output for the means-as-outcomes model, shown below in excerpted form. Note: for this example, we have clicked on Other Settings, Output Settings, and have unchecked the default "Reduced output" check box. Consequently the output below has enhanced output not found in the examples above. Click File, View output to view. Comments in italics are not part of the output.
Program: HLM 6 Hierarchical Linear and Nonlinear Modeling
Module: HLM2S.EXE (6.06.2857.2)
The data source for this run = hsb.mdm
...
The model specified for the fixed effects was:
----------------------------------------------------
Level-1 Level-2
Coefficients Predictors
---------------------- ---------------
INTRCPT1, B0 INTRCPT2, G00
MEANSES, G01
The model specified for the covariance components was:
---------------------------------------------------------
Sigma squared (constant across level-2 units)
Tau dimensions
INTRCPT1
Summary of the model specified (in equation format)
---------------------------------------------------
Level-1 Model
Y = B0 + R
Level-2 Model
B0 = G00 + G01*(MEANSES) + U0
Model. In the means-as-outcomes model, MATHACH (Y) is a function of the intercept and a residual term. The intercept is a function of the grand mean of MATHACH across the 160 schools plus a level 2 slope (the G01 b coefficient) times MEANSES, plus an error term.
Least Squares Estimates
-----------------------
sigma_squared = 41.72661
Least-squares estimates of fixed effects
----------------------------------------------------------------------------
Standard
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, B0
INTRCPT2, G00 12.712760 0.076215 166.801 7183 0.000
MEANSES, G01 5.716800 0.184286 31.021 7183 0.000
----------------------------------------------------------------------------
Least-squares estimates of fixed effects
(with robust standard errors)
----------------------------------------------------------------------------
Standard
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, B0
INTRCPT2, G00 12.712760 0.149501 85.035 7183 0.000
MEANSES, G01 5.716800 0.326862 17.490 7183 0.000
----------------------------------------------------------------------------
The least-squares likelihood value = -23600.617867
Deviance = 47201.23573
Number of estimated parameters = 1
Least squares estimates are part of the enhanced output normally suppressed when the default "Reduced output" output option is not overridden. OLS regression estimates of intercepts and slopes, unlike HLM estimates, do not treat either as random effects reflecting the random variation among schools (the level 2 units).
STARTING VALUES --------------- sigma(0)_squared = 39.14163 ....Starting values. This section, almost entirely truncated above, is part of the enhanced output when requested. Starting values are not interpreted when writing up results but may be inspected to better understand the computation process.
---------------------------------------------------- Random level-1 coefficient Reliability estimate ---------------------------------------------------- INTRCPT1, B0 0.740 ----------------------------------------------------Reliability. The higher the reliability, the less OLS and HLM estimates will diverge. As reliability of .70 or greater is still high, we would not expect large divergence in this example.
Final estimation of fixed effects:
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, B0
INTRCPT2, G00 12.649436 0.149280 84.736 158 0.000
MEANSES, G01 5.863538 0.361457 16.222 158 0.000
----------------------------------------------------------------------------
Final estimation of fixed effects
(with robust standard errors)
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, B0
INTRCPT2, G00 12.649436 0.148377 85.252 158 0.000
MEANSES, G01 5.863538 0.320211 18.311 158 0.000
----------------------------------------------------------------------------
Note that the final estimates for the intercept and meanses slope (12.65 and 5.86 respectively in the figure above) correspond to the "Estimate" for "Intercept" and "meanses" respectively in the "Estimates of Fixed Effects" table in SPSS linear mixed model output for the same model and data.
Test of the model. Above, intercepts and the slope of MEANSES are both significant, meaning that we reject the null hypothesis that there is zero difference among schools on intercepts and slopes. The intercept of 12.65 is the predicted MATHACH score when MEANSES is 0 (recall MEANSES coded 0 meant medium rather than high or low mean SES for a school). Again, the trivial differences between the ordinary and robust estimates indicates that the default normal distribution assumption for MATHACH (the outcome variable) need not be rejected.
Final estimation of variance components:
-----------------------------------------------------------------------------
Random Effect Standard Variance df Chi-square P-value
Deviation Component
-----------------------------------------------------------------------------
INTRCPT1, U0 1.62441 2.63870 158 633.51744 0.000
level-1, R 6.25756 39.15708
-----------------------------------------------------------------------------
Note that the final estimates for the variance component for "Intercpt1" and for the "level-1, R" (level 1 random error term) (2.64 and 39.16 respectively in the figure above) correspond to the "Estimate" for "Intercept[subject=id] Variance" and "Residual" respectively in the "Estimates of Covariance Parameters" table in SPSS linear mixed model output for the same model and data.
Effect size and confidence limits. Note that the variance component for the intercept is 2.64, much lower than the 8.61 value for the intercept-only model, indicating the large impact of adding MEANSES to the model. The difference is 5.97 and the ratio 5.97/8.61 = .69. The intercept-only variance component of 8,61 was the between-schools variance in MATHACH. Controlling for MEANSES added to the model reduces between-schools variance by 69%. Put another way, MEANSES explains 69% of the between-schools variance in math achievement scores. However, since the intercept is significant by the chi-square test, significant variation between school still remains. Confidence intervals on the intercept could also be constructed from the coefficients above in the same manner as in previously-discussed models.
Statistics for current covariance components model -------------------------------------------------- Deviance = 46959.446959 Number of estimated parameters = 2 Variance-Covariance components test ----------------------------------- Chi-square statistic = 157.34304 Number of degrees of freedom = 0 P-value = >.500Note that the "Deviance" estimate above corresponds to the "-2 Restricted Log Likelihood" value in the "Information Criteria" table in SPSS linear mixed model output for the same model and data.
The likelihood ratio test. Because the means-as-outcomes model and the intercept-only model both have 2 estimated parameters, degrees of freedom are 0 and while results are computed, the test is not meaningful.
Program: HLM 6 Hierarchical Linear and Nonlinear Modeling
Module: HLM2S.EXE (6.06.2857.2)
The data source for this run = hsb.mdm
The outcome variable is MATHACH
The model specified for the fixed effects was:
----------------------------------------------------
Level-1 Level-2
Coefficients Predictors
---------------------- ---------------
INTRCPT1, B0 INTRCPT2, G00
SECTOR, G01
MEANSES, G02
* SES slope, B1 INTRCPT2, G10
SECTOR, G11
MEANSES, G12
'*' - This level-1 predictor has been centered around its group mean.
The model specified for the covariance components was:
---------------------------------------------------------
Sigma squared (constant across level-2 units)
Tau dimensions
INTRCPT1
SES slope
Summary of the model specified (in equation format)
---------------------------------------------------
Level-1 Model
Y = B0 + B1*(SES) + R
Level-2 Model
B0 = G00 + G01*(SECTOR) + G02*(MEANSES) + U0
B1 = G10 + G11*(SECTOR) + G12*(MEANSES) + U1
Model. In the intercepts-and-slopes-as-outcomes model for this example, MATHACH (Y) is a function of the intercept (B0) plus the B1 coefficient times (SES - MEANSES) plus a residual error term. The B0 intercept is a function of a level-2 intercept, a level-2 slope (G01) times the level 2 variable SECTOR, plus another level-2 slope (G02) times the level 2 variable MEANSES plus an error term (U00. The level 1 B1 slope is a function of similar level 2 intercepts, slopes, and variables. In summary, the level 1 slopes and intercept are predicted from level 2 variables.
Sigma_squared = 36.70313
Tau
INTRCPT1,B0 2.37996 0.19058
SES,B1 0.19058 0.14892
Tau (as correlations)
INTRCPT1,B0 1.000 0.320
SES,B1 0.320 1.000
Sigma-squared and tau. The tau values are a variance-covariance matrix, where the covariance of the SES intercept and slope error terms for the 160 schools in this example is .19 and the corresponding correlation is .32. On the main diagonal, the variance estimate for the intercepts is 2.37 and for the slopes is .15.
Intraclass correlation (ICC). With two random effects (here the intercept and slope of centered ses), the total of variance components is the within-group variance (sigma-squared = 36.70313) plus the level 2 between-group variance terms: 2.37996 (the level 2 intercept variance estimate) + 0.14892 (the level 2 slope variance estimate) plus twice the covariance: 2*0.19058. For these data, the total variance sums to 39.61327. For a variance components model, constrained to have zero covariance of the random effect error terms, there would be no 2*cov term. For other models the covariance must be taken into account because the variance components are no longer additive in a simple way. The ICC is the between-groups effect on the intercept of the outcome variable (2.37996) divided by total variance (39.61327) = .06. We may say that 6% of the variance in math achievement is attributable to the between-groups effect on mean math achievement.
Did modeling level 2 effects make a difference? One may calculare the total level 2 effect as the sum of the level 2 intercept effect (2.37996) plus the level 2 slope effect (0.14892) plus twice the covariance of intercept and slope error terms at level 2 (2*0.19058), this quantity divided by the total of variance components as calculated above for ICC (39.61327). Making this calculation yields the value .0735. We may say that taking level 2 into account explained 7.35% of the total variance in math achievement. This is percentage often is considered a better effect size measure for HLM than is the ICC. Note that in variance components models the 2*cov term drops out.
----------------------------------------------------
Random level-1 coefficient Reliability estimate
----------------------------------------------------
INTRCPT1, B0 0.733
SES, B1 0.073
----------------------------------------------------
Reliability. The reliability (see above) is high for the intercept but low for the slope. Below, in the "Final estimation of variance components" table, we find that the between schools slope is not significant.
...
Final estimation of fixed effects:
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, B0
INTRCPT2, G00 12.096006 0.198734 60.865 157 0.000
SECTOR, G01 1.226384 0.306272 4.004 157 0.000
MEANSES, G02 5.333056 0.369161 14.446 157 0.000
For SES slope, B1
INTRCPT2, G10 2.937981 0.157135 18.697 157 0.000
SECTOR, G11 -1.640954 0.242905 -6.756 157 0.000
MEANSES, G12 1.034427 0.302566 3.419 157 0.001
----------------------------------------------------------------------------
Final estimation of fixed effects
(with robust standard errors)
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, B0
INTRCPT2, G00 12.096006 0.173699 69.638 157 0.000
SECTOR, G01 1.226384 0.308484 3.976 157 0.000
MEANSES, G02 5.333056 0.334600 15.939 157 0.000
For SES slope, B1
INTRCPT2, G10 2.937981 0.147620 19.902 157 0.000
SECTOR, G11 -1.640954 0.237401 -6.912 157 0.000
MEANSES, G12 1.034427 0.332785 3.108 157 0.003
----------------------------------------------------------------------------
Parameter estimates. The intercept-and-slopes-as-outcomes model used the level 2 variables SECTOR and MEANSES to predict the level-1 intercept (B0) and the level 1 slope of group-centered SES (B1). In the table above we see that SECTOR and MEANSES are both significant predictors of both B0 and B1. SECTOR is the public vs. parochial school binary variable. MEANSES is a trichotomized measure of whether a school is low, medium, or high on average SES of its students. The significant p values for SECTOR mean that public and parochial schools differ significantly on both intercepts (average math achievement controlling for other variables in the model) and slopes (strength of the relation of SES to math achievement controlling for other variables in the model). Likewise, the significant p values for MEANSES indicate that schools with different level 2 SES averages also differ in these same ways.
In the "Final Estimation of Fixed Effects" table above, there are two sets of coefficients:
All these estimates are tested to be significant, as indicated in the "P-value" column.
Final estimation of variance components:
-----------------------------------------------------------------------------
Random Effect Standard Variance df Chi-square P-value
Deviation Component
-----------------------------------------------------------------------------
INTRCPT1, U0 1.54271 2.37996 157 605.29503 0.000
SES slope, U1 0.38590 0.14892 157 162.30867 0.369
level-1, R 6.05831 36.70313
-----------------------------------------------------------------------------
Variance components. The variance components table has one row for each random effect. Here the random effects are the error term for the level-2 intercept (U0), the error term for the level 2 slope (U1), and the error term for the level 1 equation (R).
Statistics for current covariance components model -------------------------------------------------- Deviance = 46501.875643 Number of estimated parameters = 4 Variance-Covariance components test ----------------------------------- Chi-square statistic = 614.91783 Number of degrees of freedom = 2 P-value = 0.000Likelihood ratio test. Deviance for the intercept-and-slopes-as-outcomes model is significantly lower than for the intercept-only model by the likelihood ratio test, indicating it is a significantly better model.
Test of homogeneity of level-1 variance ---------------------------------------- Chi-square statistic = 244.08638 Number of degrees of freedom = 159 P-value = 0.000When significant, as above for the intercepts-and-slopes-as-outcomes model just discussed, the researcher rejects the null hypothesis that level 1 variance of residuals is homogenous. Lack of homogeneity may be due to a variety of causes including outliers, non-normal (heavily kurtotic) data, omitting one or more important level 1 variables, or treating an included level 1 variable as a fixed effect when it is not.
SPSS. The HLM heterogeneity of variance test does not tell the researcher which other predictors may be affecting residual variance. This can be explored using SPSS to view the variances of the OLS residuals by gender. Select Analyze, Regression, Linear; set Dependent = MATHACH, Independent = SES, Selection variable = FEMALE with Rule FEMALE = 0 in a first run, the FEMALE = 1 in a second run. In the ANOVA table output, residual sum of squares for males (FEMALE = 0) is 148,250 and for females (FEMALE = 1) it is 144,022. (Alternatively, the Save button would allow residuals to be saved on each run and then their variances could be viewed under the Descriptive statistics option.)
HLM heterogenous variance modeling. Heterogenous variance models are any of the other types of models with the heterogenous sigma option selected. For instance, one could adapt the intercepts-and-slopes-as-outcomes model above to model the level 1 error term using FEMALE. To do this, in HLM select Other Settings, Estimation Settings from the menu and then click the Heterogenous Sigma^2 button. (Recall Sigma2 reflects within group variance.) This brings up the dialog below.
The model. Back in the main HLM interface, a variance equation for the level 1 residual is added to the level 1 equation section. Otherwise the intercepts-and-slopes-as-outcomes model is unchanged.
The deviance for the intercepts-and-slopes-as-outcomes model without heterogenous variance modeled was 46501.88. Asking under Other Settings, Hypothesis Testing, for a model comparison as described in previous sections, the output for the heterogenous variance modified model contains slightly different parameter estimates which are significant by the model comparison test:
Statistics for current covariance components model -------------------------------------------------- Deviance = 46482.093344 Number of estimated parameters = 11 Model comparison test ----------------------------------- Chi-square statistic = 19.78230 Number of degrees of freedom = 7 P-value = 0.006
The model above may be run unconstrained, then SECTOR can be constrained to be equal for Male and Female. To do this, Other Settings, Estimation Settings is chosen from the HLM menu, then the "Constraint of fixed effects" button is clicked, bring up the dialog below. A "0" entry leaves a parameter unconstrained. A pair of "1" entries forces an equality constraint. Here, the level 2 determinants of the slopes of the level 1 variables Male and Female are constrained to be equal.
In the unconstrained model, the level 2 slopes the way SECTOR influences the level 1 slopes for FEMALE and MALE differ:
Final estimation of fixed effects
(with robust standard errors)
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For FEMALE slope, B1
INTRCPT2, G10 10.684432 0.298122 35.839 158 0.000
SECTOR, G11 2.932540 0.446512 6.568 158 0.000
For MALE slope, B2
INTRCPT2, G20 12.174859 0.322616 37.738 158 0.000
SECTOR, G21 2.597771 0.487027 5.334 158 0.000
----------------------------------------------------------------------------
In the constrained model, the slopes are the same. The level 2 G21 slope is not displayed as it is equal to the G11 slope.
Final estimation of fixed effects
(with robust standard errors)
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For FEMALE slope, B1
INTRCPT2, G10 10.723664 0.295717 36.263 158 0.000
SECTOR, G11 * 2.804823 0.417646 6.716 158 0.000
For MALE slope, B2
INTRCPT2, G20 12.103608 0.313462 38.613 159 0.000
----------------------------------------------------------------------------
The "*" gammas have been constrained. See the table on the header page.
This was the deviance computed for the unconstrained model. It is entered as the comparison for the constrained model further below.
Statistics for current covariance components model -------------------------------------------------- Deviance = 47012.437355 Number of estimated parameters = 4The unconstrained and constrained variance components are very close.
Final estimation of variance components (unconstrained model):
-----------------------------------------------------------------------------
Random Effect Standard Variance df Chi-square P-value
Deviation Component
-----------------------------------------------------------------------------
FEMALE slope, U1 2.41260 5.82064 121 481.99916 0.000
MALE slope, U2 2.64370 6.98917 121 483.25462 0.000
level-1, R 6.22438 38.74285
-----------------------------------------------------------------------------
Final estimation of variance components (constrained model):
-----------------------------------------------------------------------------
Random Effect Standard Variance df Chi-square P-value
Deviation Component
-----------------------------------------------------------------------------
FEMALE slope, U1 2.40847 5.80071 121 484.11557 0.000
MALE slope, U2 2.63048 6.91943 121 483.35444 0.000
level-1, R 6.22449 38.74426
-----------------------------------------------------------------------------
This was the deviance computed for the unconstrained model. It is entered as the comparison for the constrained model further below.
Statistics for current covariance components model
--------------------------------------------------
Deviance = 47012.437355
Number of estimated parameters = 4
The constrained and unconstrained models have the same number of estimated parameters, so the likelihood ratio test has 0 degrees of freedom, making the test unavailable. However, we can see the deviance is practically identical. As assuming equal level 2 slopes is more parsimonious than assuming different ones, the researcher may conclude that there is slope invariance across sectors.
Statistics for current covariance components model
--------------------------------------------------
Deviance = 47014.952691
Number of estimated parameters = 4
Variance-Covariance components test
-----------------------------------
Chi-square statistic = 2.51534
Number of degrees of freedom = 0
P-value = >.500
Below, the the "Level 1 equation graphing" dialog, we choose a random sample of 10% of the 160 schools.
Below, lines in blue are SECTOR = 0 = public schools, while those in red are SECTOR = 1 = parochial schools. In this random sample, there are 6 public and 10 parochial schools, hence a total of 16 lines in the graph. The lines depict graphically how, as the level 1 variable centered SES increases within any given school, math achievement goes up. The slopes of the blue public school lines are steeper than the red line parochial school slopes, showing as discussed above that the within-school relation of SES to MATHACH is stronger for public schools.
Below, the resulting graph shows that the same pattern of stronger relation (slope) of SES to MATHACH persists: at all selected levels of MEANSES, the slope of SES is greater for public schools (the blue lines).
From the same "Model graphs" dialog, we can set the X focus as the level 2 variable SECTOR and the Z focus as the level 2 variable MEANSES, generating the graph below. This bar chart shows that at any percentile level of MEANSES (25th, 50th, 75th), math achievement is higher for parochial schools (SECTOR = 1) than for public schools.
In the resulting graph, the Y axis represents the MATHACH residuals. The X axis is not meaningful, only representing the 16 random schools selected in each of two runs of this graph. In the box-and-whisker plot for any school, the box is the interquartile range and the whiskers are the distance to the minimum and maximum residual values. These box-and-whisker plots show than for this model, residuals (error) are generally greater for public than for parochial schools, and dispersion of error is also greater.
The resulting graph below shows, for the 16 randomly selected schools, than confidence limits on B0 do not differ markedly by sector.
The resulting graph below shows the individual-level relation of SES to MATHACH, by SECTOR. Points thus represent students, with blue points for public school and red points for parochial school students. It is possible to ask for one such graph per school, bhut here the first 10 are combined in one graph. The graph shows that math achievement scores tend to be higher for parochial schools. The relationship of SES to MATHACH is not very strong (as would be reflected by points being on a line rather than forming a cloud) for either sector, but is stronger for the public than parochial sector - leading to the expectation of high B1 slopes for SES in the public than the parochial sector.
There is and must be a common link field, in this case "schoolid", linking all three levels. That is, "schoolid" is the level 3 link variable. There is and must also be a second common link field linking levels 1 and 2. Here "childid" is the level 2 link variable.
Warning: In HLM, the level-1 and level-2 files must be sorted in the same order of level-2 ID nested within level-3 ID. That is, in this case, the schoolid order in the level 3 file (EG3.SAV) must be the schoolid order in the level 1 and level 2 files (EG1.SAV and EG2.SAV). The childid order in the level 2 file must be the childid order in the level 1 file. Failure to sort properly will lead to incorrect results.
Descriptive Statistics N Minimum Maximum Mean Std. Deviation math 7230 -5.22 5.77 -.5369 1.53470 Valid N (listwise) 7230
When the researcher clicks the "Make MDM" button, a file called HLM3MDM.STS is created containing descriptive statistics on the data. The "Check Stats" button should be clicked to see if data look right and if the sample size seems low (this may be due to not sorting properly). After clicking "Done" on the "MAKE MDM - HLM3" dialog, the HLM working file is created under the name given by the researcher. From this point on, the .SAV files are no longer used.
In the HLM intercept-only model, the intercept is modeled as a random coefficient. Specifically, at level 1 (across measurement periods), the dependent variable MATH is a function of the intercept and an error term. At level 2, the level 1 intercept is a function of the level 2 intercept and an error term. At level 3, the level 2 intercept is a function of the level 3 intercept and an error term. The level 1 intercepts are assumed to vary randomly across the level 2 units, which in this example are the 1,721 children (sets of measurement periods by child) within schools. The level 2 intercepts are assumed to vary randomly across the level 3 units, which here are the 60 schools.
Program: HLM 6 Hierarchical Linear and Nonlinear Modeling Module: HLM3S.EXE (6.06.2857.2) The data source for this run = C:\HLM6 Student Examples\Chapter4\EG.MDM The command file for this run = C:\HLM6 Student Examples\Chapter4\EG0.hlm The maximum number of level-1 units = 7230 The maximum number of level-2 units = 1721 The maximum number of level-3 units = 60 The outcome variable is MATHThere were 60 schools with 1,721 children and 7,230 measurements.
... Summary of the model specified (in equation format) --------------------------------------------------- Level-1 Model Y = P0 + E Level-2 Model P0 = B00 + R0 Level-3 Model B00 = G000 + U00 ... Sigma_squared = 1.52393 Standard Error of Sigma_squared = 0.02900Sigma-squared represents the level 1 variance of the intercept. It is the variance of the level 1 error term, E.
Tau(pi) INTRCPT1,P0 0.57038 Tau(pi) (as correlations) INTRCPT1,P0 1.000 Standard Errors of Tau(pi) INTRCPT1,P0 0.03338Tau-pi represents the level 2 variance of the intercept. It is the variance of the level 2 error term, R0, in the model specification in output above. As such it contributes to the estimated slope at level 1.
---------------------------------------------------- Random level-1 coefficient Reliability estimate ---------------------------------------------------- INTRCPT1, P0 0.604 ----------------------------------------------------The level 1 reliability is the reliability of the level 1 intercepts across the 1,721 children (recall children are the level 2 units). The larger the number of level 1 units per level 2 unit, and the larger the level 1 variance component (sigma-squared = 1.52) relative to the level 2 variance component (tau-pi = .57), the closer to 1.0 will be the reliability. In this example, the level 1 variance component is large relative to the level 2 variance component (1.52/.57 = 2.67) but there are not very many level 1 units per level 2 unit (7230/1721 = 4.2). The former ratio drives reliability up toward 1.0 and the latter ratio drives reliability down, resulting in overall moderate reliability of .604.
Tau(beta)
INTRCPT1
INTRCPT2,B00
0.31767
Tau(beta) (as correlations)
INTRCPT1/INTRCPT2,B00 1.000
Standard Errors of Tau(beta)
INTRCPT1
INTRCPT2,B00
0.06636
Tau-beta represents the level 3 variance of the intercept. It is the variance of the level 3 error term, U00 in the model specification in output above. As such it contributes to the estimate of the slope at level 2.
The total variance is sigma-squared plus tau-pi plus tau-beta = 1.524 + .570 + .318 = 2.412. The variance in MATH score attributable to variance in YEAR (the level 1 unit) is thus 1.524/2.412 = 63.2%. Most of the variance is due to children within schools improving MATH score from measurement year to measurement year. The variance in score attributable to variation at level 2 (the child level) is .570/2.412 = 23.6%. The variance in score attibutable to variation at level 3 (the school level) is .318/2.412 = 13.2%.
---------------------------------------------------- Random level-2 coefficient Reliability estimate ---------------------------------------------------- INTRCPT1/INTRCPT2, B00 0.871 ----------------------------------------------------The level 2 reliability is the reliability of the level 2 intercepts across the 60 schools (recall schools are the level 3 units). Reliability will approach 1.0 as the ratio of the level 2 variance component is large relative to the level 3 variance component (tau-pi/tau-beta = .57/.32 = 1.78) and when the number of level 2 units is large relative to level 3 units adjusting for level 2 reliability ((1721 children/60 schools)*.604 = 17.3). Here while the former ratio is lower than for level 1 reliability, the latter ratio is much larger, resulting in the stronger reliability estimate of .871.
...
Final estimation of fixed effects:
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, P0
For INTRCPT2, B00
INTRCPT3, G000 -0.510018 0.077970 -6.541 59 0.000
----------------------------------------------------------------------------
Final estimation of fixed effects
(with robust standard errors)
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, P0
For INTRCPT2, B00
INTRCPT3, G000 -0.510018 0.077964 -6.542 59 0.000
----------------------------------------------------------------------------
The estimated MATH intercept is significant at -.51, representing the mean MATH score when no predictors are in the model.
Final estimation of level-1 and level-2 variance components:
------------------------------------------------------------------------------
Random Effect Standard Variance df Chi-square P-value
Deviation Component
------------------------------------------------------------------------------
INTRCPT1, R0 0.75524 0.57038 1661 4253.88860 0.000
level-1, E 1.23447 1.52393
------------------------------------------------------------------------------
Final estimation of level-3 variance components:
------------------------------------------------------------------------------
Random Effect Standard Variance df Chi-square P-value
Deviation Component
------------------------------------------------------------------------------
INTRCPT1/INTRCPT2, U00 0.56362 0.31767 59 573.08980 0.000
------------------------------------------------------------------------------
Above, random effect variance components are shown. The level 1 variance component is .57, the level 2 variance component is 1.52, and the level 3 variance component is .32. Total variance is thus 2.41. The largest variance component is 1.52 (63% of the total), showing that the greatest variance in MATH is at level 2, which means across children within schools. The level 1 variance component is .57 (24% of the total), reflecting variance among measurement time periods across children within schools. The smallest variance component is .32 (13% of the total), showing that between-schools variance (the level 3 component) explains the least variance in MATH. Since schools explains relatively little of the total variance in MATH, a more complex model with predictor variables beyond the intercept is called for.
Statistics for current covariance components model -------------------------------------------------- Deviance = 25305.980829 Number of estimated parameters = 4The deviance is a measure of model fit. The intercept-only (null model) deviance is compared with the deviance in other more complex models, with lower deviance being better fit. The likelihood ratio test, discussed above, tests the significance of the difference in deviance values between models.
Program: HLM 6 Hierarchical Linear and Nonlinear Modeling Module: HLM3S.EXE (6.06.2857.2) The data source for this run = C:\HLM6 Student Examples\Chapter4\EG.MDM ... The outcome variable is MATH Summary of the model specified (in equation format) --------------------------------------------------- Level-1 Model Y = P0 + P1*(YEAR) + E Level-2 Model P0 = B00 + R0 P1 = B10 + R1 Level-3 Model B00 = G000 + U00 B10 = G100 + U10 Sigma_squared = 0.30148 Standard Error of Sigma_squared = 0.00660Above, sigma-squared is level 1 variance within level 2 units (children). In the intercept-only model sigma-squared was 1.52, but now it has dropped to .30, a relatively small magnitude. That is, sigma-squared is the variance in the level 1 error term once YEAR is placed in the level 1 equation. The variance of MATH by observation YEAR is small within the measures for any given child within any given school, once the linear effect of measurement YEAR is controlled.
Tau(pi)
INTRCPT1,P0 0.64049 0.04676
YEAR,P1 0.04676 0.01122
Tau(pi) (as correlations)
INTRCPT1,P0 1.000 0.551
YEAR,P1 0.551 1.000
...
Tau-pi is now a 2-by-2 matrix because there are two random effects modeled at level 2: the level 1 intercept and YEAR. The variance associated with the level 1 intercept (.64) is much greater than the variance associated with YEAR (.01). The covariance of intercept error with error for slope of YEAR was .05 in the level 2 model.
...
----------------------------------------------------
Random level-1 coefficient Reliability estimate
----------------------------------------------------
INTRCPT1, P0 0.839
YEAR, P1 0.190
----------------------------------------------------
Tau(beta)
INTRCPT1 YEAR
INTRCPT2,B00 INTRCPT2,B10
0.16531 0.01705
0.01705 0.01102
Tau(beta) (as correlations)
INTRCPT1/INTRCPT2,B00 1.000 0.399
YEAR/INTRCPT2,B10 0.399 1.000
Tau-beta is also a 2-by-2 matrix because it too models two random effects: level 3 models the two level 2 intercepts - one for the level 1 intercept and one for the level 1 slope of YEAR. Again, the variation associated with the intercepts is much greater than that for YEAR. Here at level 3, the correlation of intercepts and slopes is moderate (.399) and weaker than at level 2 (where it was .055). That is, the tendency of high average MATH score to be associated with a high effect of measurement YEAR was more pronounced at the child level (level 2) than at the school level (level 3).
Calculating the effect of adding YEAR to the null model. Sigma-squared for the intercept-only model was 1.52, representing the variance of the level 1 error term when there were no predictors. Now for the unconditional growth model, it is .30. The reduction is 1.22. The reduction divided by the intercept-only sigma-squared = 1.22/1.52 = .802. That is, 80.2% of the level 1 variance in the intercept-only model is accounted for by adding YEAR to the level 1 equation.
Variance proportions. The remaining variation in MATH score after linear effects of YEAR at level 1 are controlled are .302 for level 1 (across measurement years), .640 for level 2 (child level), and .165 for level 3 (school level). This is total variance of 1.107. Dividing the level variance component by total gives variance proportions of .272 for level 1, .578 for level 2, and .149 for level 3. We may say that the variance in MATH score after the linear effect of measurement YEAR is controlled is 27.2% associated with nonlinear and residual effects of YEAR, the level 1 unit; 57.8% associated with variation among children within schools; and 14.9% associated with between-school variation, in this model.
----------------------------------------------------
Random level-2 coefficient Reliability estimate
----------------------------------------------------
INTRCPT1/INTRCPT2, B00 0.821
YEAR/INTRCPT2, B10 0.786
----------------------------------------------------
Final estimation of fixed effects:
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, P0
For INTRCPT2, B00
INTRCPT3, G000 -0.779309 0.057829 -13.476 59 0.000
For YEAR slope, P1
For INTRCPT2, B10
INTRCPT3, G100 0.763029 0.015263 49.993 59 0.000
----------------------------------------------------------------------------
Final estimation of fixed effects
(with robust standard errors)
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, P0
For INTRCPT2, B00
INTRCPT3, G000 -0.779309 0.057830 -13.476 59 0.000
For YEAR slope, P1
For INTRCPT2, B10
INTRCPT3, G100 0.763029 0.015260 50.000 59 0.000
----------------------------------------------------------------------------
The effect of YEAR on MATH score is significant at the .000 level. For each unit increase in measurement YEAR, the logit of MATH increases .763.
Final estimation of level-1 and level-2 variance components:
------------------------------------------------------------------------------
Random Effect Standard Variance df Chi-square P-value
Deviation Component
------------------------------------------------------------------------------
INTRCPT1, R0 0.80030 0.64049 1661 13679.62589 0.000
YEAR slope, R1 0.10595 0.01122 1661 2132.50756 0.000
level-1, E 0.54907 0.30148
------------------------------------------------------------------------------
The slope of YEAR is significant. However, the level 1 intercept, with a variance component of .64, is the largest component, indicating that variability on MATH within schools is substantial even after controlling for YEAR (that is, setting YEAR at its mean of 3.5). Likewise, the variance component for the level 1 error term is also appreciable (.30). Both suggest the need for additional variables in the model.
Final estimation of level-3 variance components:
------------------------------------------------------------------------------
Random Effect Standard Variance df Chi-square P-value
Deviation Component
------------------------------------------------------------------------------
INTRCPT1/INTRCPT2, U00 0.40658 0.16531 59 488.30922 0.000
YEAR/INTRCPT2, U10 0.10498 0.01102 59 377.43020 0.000
------------------------------------------------------------------------------
At level 3, above, the variance component for the intercept is much larger than for YEAR, again suggesting the need for additional variables in the model.
Statistics for current covariance components model -------------------------------------------------- Deviance = 16326.231292 Number of estimated parameters = 9 Model comparison test ----------------------------------- Chi-square statistic = 8979.74871 Number of degrees of freedom = 5 P-value = 0.000Deviance is reduced from 25306 in the intercept-only model to 16326 in the unconditional growth model here. By the likelihood ratio test (model chi-square test), this difference is significant at the .000 level, confirming that adding YEAR to the model leads to better fit with the data.
Program: HLM 6 Hierarchical Linear and Nonlinear Modeling Module: HLM3S.EXE (6.06.2857.2) The outcome variable is MATH ... Summary of the model specified (in equation format) --------------------------------------------------- Level-1 Model Y = P0 + P1*(YEAR) + E Level-2 Model P0 = B00 + B01*(BLACK) + B02*(HISPANIC) + R0 P1 = B10 + B11*(BLACK) + B12*(HISPANIC) + R1 Level-3 Model B00 = G000 + G001(LOWINC) + U00 B01 = G010 B02 = G020 B10 = G100 + G101(LOWINC) + U10 B11 = G110 B12 = G120 ... Sigma_squared = 0.30162 Standard Error of Sigma_squared = 0.00660Again, sigma-squared reflects the level 1 variance in the intercept, which in turn is the mean value of MATH when predictors are 0 (which for YEAR corresponds to YEAR 3.5 since it is centered).
Tau(pi)
INTRCPT1,P0 0.62231 0.04657
YEAR,P1 0.04657 0.01106
Tau(pi) (as correlations)
INTRCPT1,P0 1.000 0.561
YEAR,P1 0.561 1.000
Standard Errors of Tau(pi)
INTRCPT1,P0 0.02451 0.00491
YEAR,P1 0.00491 0.00196
----------------------------------------------------
Random level-1 coefficient Reliability estimate
----------------------------------------------------
INTRCPT1, P0 0.835
YEAR, P1 0.188
----------------------------------------------------
Tau-pi is is a variance/covariance matrix for the two random effects modeled at level 2: the level 1 intercept and YEAR. The variance associated with the level 1 intercept (.62) is still much greater than the variance associated with YEAR (.01). The reliability for the intercept is much greater than for the slope of YEAR. The covariance of intercept and slope of YEAR error was .05 in the level 2 model.
Tau(beta)
INTRCPT1 YEAR
INTRCPT2,B00 INTRCPT2,B10
0.07808 0.00082
0.00082 0.00798
Tau(beta) (as correlations)
INTRCPT1/INTRCPT2,B00 1.000 0.033
YEAR/INTRCPT2,B10 0.033 1.000
Standard Errors of Tau(beta)
INTRCPT1 YEAR
INTRCPT2,B00 INTRCPT2,B10
0.01991 0.00441
0.00441 0.00194
----------------------------------------------------
Random level-2 coefficient Reliability estimate
----------------------------------------------------
INTRCPT1/INTRCPT2, B00 0.702
YEAR/INTRCPT2, B10 0.735
----------------------------------------------------
Tau-beta is also variance-covariance matrix, but as modeled at level 3 for the intercept and YEAR intercepts at level 2. Again, the variation associated with the intercepts (.078) is much greater than that for the level 2 YEAR intercept (.008). Here at level 3, the correlation of intercept and slope error is weak (.033) compared to level 2 (where it was .561), indicating that the tendency of high average MATH score to be associated with a high effect of measurement YEAR was much more pronounced at the child level (level 2) than at the school level (level 3).
Calculating the effect of adding BLACK, HISPANIC, and LOWINC to the unconditional growth model. Sigma-squared for the unconditional growth model was 0.30, representing the variance of the level 1 error term when only the linear effect of YEAR was controlled. Now for the conditional growth model, it is still .30. The reduction is close to zero. That is, none of the level 1 variance remaining in the unconditional growth model is accounted for by adding BLACK, HISPANIC, and LOWINC to the model. Note that in a full analysis, variables would have been added one at a time in each subsequent model.
Variance proportions. The remaining variation in MATH score after linear effects of YEAR at level 1 as well as the effects of BLACK and HISPANIC at level 2 and LOWINC at level 3 are controlled are .302 for level 1 (across measurement years), .622 for level 2 (child level), and .078 for level 3 (school level). This is total variance of 1.002. Dividing the level variance component by total gives variance proportions of .301 for level 1, .621 for level 2, and .078 for level 3. We may say that the variance in MATH score after other variables are controlled is 30.1% associated variation among measurement years, the level 1 unit; 62.1% associated with variation among children within schools; and 7.8% associated with between-school variation, in this model. The main impact of adding BLACK, HISPANIC, and LOWINC to the model is to reduce the between-school variance proportion from 14.9% to 7.8%.
...
Final estimation of fixed effects
(with robust standard errors)
----------------------------------------------------------------------------
Standard Approx.
Fixed Effect Coefficient Error T-ratio d.f. P-value
----------------------------------------------------------------------------
For INTRCPT1, P0
For INTRCPT2, B00
INTRCPT3, G000 0.140628 0.113814 1.236 58 0.222
LOWINC, G001 -0.007578 0.001396 -5.428 58 0.000
For BLACK, B01
INTRCPT3, G010 -0.502091 0.076842 -6.534 1718 0.000
For HISPANIC, B02
INTRCPT3, G020 -0.319381 0.081918 -3.899 1718 0.000
For YEAR slope, P1
For INTRCPT2, B10
INTRCPT3, G100 0.874501 0.037287 23.453 58 0.000
LOWINC, G101 -0.001369 0.000499 -2.744 58 0.009
For BLACK, B11
INTRCPT3, G110 -0.030918 0.022274 -1.388 1718 0.165
For HISPANIC, B12
INTRCPT3, G120 0.043085 0.024368 1.768 1718 0.077
----------------------------------------------------------------------------
The slope of YEAR is .87, meaning that as measurement YEAR increases 1 unit, the logit of MATH increases by .87 - a significant average growth rate. At level 3, which is the school level, BLACK and HISPANIC both significantly affect the intercept term, which reflects average MATH score when other variables in the model are controlled. At the school level, BLACK and HISPANIC do not significantly affect the slope term for YEAR, which reflects the rate of increase in MATH score across measurement YEARs. In contrast, LOWINC significantly affects both intercept and slope (both average score and rate of increase).
Final estimation of level-1 and level-2 variance components:
------------------------------------------------------------------------------
Random Effect Standard Variance df Chi-square P-value
Deviation Component
------------------------------------------------------------------------------
INTRCPT1, R0 0.78886 0.62231 1659 13364.57298 0.000
YEAR slope, R1 0.10518 0.01106 1659 2126.73092 0.000
level-1, E 0.54920 0.30162
------------------------------------------------------------------------------
Again, the variance component for the level 1 intercept as modeled at the child level (level 2) is the largest component, though the variance component for YEAR is significant. There is also sizeable residual variation, reflecting the need for additional variables in the model.
Final estimation of level-3 variance components:
------------------------------------------------------------------------------
Random Effect Standard Variance df Chi-square P-value
Deviation Component
------------------------------------------------------------------------------
INTRCPT1/INTRCPT2, U00 0.27943 0.07808 58 254.96395 0.000
YEAR/INTRCPT2, U10 0.08935 0.00798 58 277.26967 0.000
------------------------------------------------------------------------------
The variance components at the school level (level 3) are significant for both the intercept and YEAR terms when YEAR is 0 (recall YEAR was centered, so 0 corresponds to measurement YEAR 3.5). The table above reflects the effects of the school level on intercepts at level 2, which in turn affect the level 1 intercept and slope of YEAR. Differences among schools significantly affect both the intercept (mean MATH score) and the slope of YEAR (rate of score increase over measurement periods). The level 3 variance components are controlling for LOWINC and are lower as a result compared to the unconditional linear growth model previously discussed, where the intercept component was .165 and the YEAR component was .011 .
Statistics for current covariance components model -------------------------------------------------- Deviance = 16239.207232 Number of estimated parameters = 15 Model comparison test ----------------------------------- Chi-square statistic = 9066.77360 Number of degrees of freedom = 11 P-value = 0.000Above, the likelihood ratio test shows the conditional growth model to be significantly better than the null model. For the intercept-only three-level model, deviance was 25306. For the unconditional growth model, with only YEAR as a level 1 predictor, deviance was 16326. Lower deviance is better fit. Adding HISPANIC, BLACK, and LOWINC to the model has only improved fit marginally. Nonetheless, a separate likelihood ratio test, not shown, shows that the improvement is significant.
Also note that if one has a binary variable (such as "sector" in the school examples above), HLM will treat it as a covariate. SPSS results will be similar if "sector" is entered as a covariate, but if entered as a factor, signs will be reversed. That is, for factors, SPSS predicts the lower value (sector=0) and makes the higher value (sector=1) the reference. For covariates, the highest value (sector=1) is predicted. Similar differences will for multinomial variables entered as factors in SPSS.
In SPSS, open both the level 1 and level 2 .SAV files used to create the .MDM file. Make the level 2 file the active one. In SPSS, select Data, Merge, Add Variable. In the ensuing dialog, select the level 1 file as the one to merge with. Then check "Match cases on key variables". Then select the radio button reading "Active dataset is keyed table." Set the key variable to the id variable. Note that the data must have been sorted ascending on the key (id) variable.