|
|
It is a very good idea to have an alternative theory to contrast with your main theory. If, for instance, you are using diffusion theory, you ought to be able to imagine "not-diffusion theory". There ought to be some alternative theory which points to the importance of variables not the ones expected to be important by diffusion theory. That is, your main theory must be falsifiable. A theory which cannot be falsified is not a social science theory but is instead merely a descriptive vocabulary, no better than other general-purpose vocabularies.
However, it is certainly true that normally if one has two scales, one expects items within each scale to correlate highly with each other. I would say the standard approach is to show the items in the scale meet the .7 level of Cronbach's alpha. There are lots of published works which never seek additional rigor, but it is best to go beyond alpha, which demonstrates convergent validity but not discriminant validity.
The next most rigorous criterion is to say that if one has two scales, the items of each should correlate more highly among themselves than across concepts. Cronbach's alpha does not address this and thus is a minimal test. I think the most popular method to analyze cross-loadings is to show the concepts have an acceptable fit in confirmatory factor analysis of the measurement model in a structural equation modeling program like LISREL or AMOS. Unlike conventional factor analysis, CFA will (1) give a single measure yes- no fit criterion; and (2) generate modification indexes and standard errors for adding/subtracting arrows to the model, and bottom line, one could modify the measurement model one indicator at a time and with luck one would wind up with a two-concept model which still had acceptable fit. This is discussed further on my validity page.
An alternative method, using conventional factor analysis to see if indicators sort themselves out on separate factors, also is perfectly valid and more rigorous than alpha, though it does not generate the yes-no single-measure fit criterion of SEM and is not considered as sophisticated an approach. When I use this traditional factor approach, I usually apply it to all items in the instrument, but probably similar results would ensue.
Copyright 1998, 2008 by G. David Garson.
Last update: 11/15/08.