RESEARCH DESIGN

Overview

Leaving aside non-experimental research, quantitative research designs fall into two broad classes: experimental and quasi-experimental. Experimental studies are characterized by the ability to randomize subjects into treatment and control groups. This randomization goes a long way toward controlling for variables which are not included explicitly in the study. Comparison groups in quasi-experimental research are not true randomized control groups as in experimental research. Quasi-experimental research therefore has to control for confounding variables by adding them explicitly in various multivariate statistical techniques which adjust estimates of the relation of causal variables with dependent variables for their correlation with control variables. For this reason, quasi-experimental studies are sometimes called "correlational designs".

Sometimes the term "research design" is equated with the broad study of the scientific method, including aspects of theory construction, hypothesis formation, data collection, and even data analysis. Works on research design by Creswell (2008, 2012), for instance, treat such broad topics as scientific philosophy (ex., pragmatism vs. postpositivism) and methodology of data collection (ex., structured questionnaires versus open-ended interview). To take a second example, Leedy & Ormrod (2009) treat "research design" in terms of building theory, operationalizing definitions, forming hypotheses, using databases, and other phases of the research process. In a third example, for Mitchell & Jolley (2012), "research design" is generating hypotheses, reviewing the literature, operationalizing variables, using descriptive and correlational methods to describe the data, survey research, validity, and experiments.

In contrast to these broader usages of the term, in this volume "research design" is treated under a narrower definition having to do with how measurement should be structured so that effects on the dependent variable may be observed and valid inferences made. While there is some overlap with the broader usages of the term, here research design focuses on the interrelationships among subject selection and grouping, exposure to the dependent variable (treatment in experimental studies), and time of measurement.

For instance, not much can be concluded from one-point-in-time studies of a single group. It is much more informative to have a comparison or control group, and better yet to have measurements before (pretest) and after (posttest) the introduction of the causal variable (the treatment in experimental studies or the change in the causal variable(s) in quasi-experimental studies). It is best of all to have multiple pretests and posttests (time series). These and other considerations of research design are discussed below.

Research design is largely independent of the choice of methods of data collection. Interviewing and survey research, for instance, may be used in experimental, quasi-experimental, and non-experimental research. Similarly, analysis of variance (ANOVA) studies may be experimental or quasi-experimental even though this procedure originated in the experimental research. More on research design may be found in the separate Statistical Associates "Blue Book" volumes on univariate and multivariate GLM (GLM implements analysis of variance). In practical terms, however, some methods of data collection, such as case studies, are used almost exclusively in non-experimental designs.

The full content is now available from Statistical Associates Publishers. Click here.

Below is the unformatted table of contents.

Table of Contents
Overview	6
Key Concepts and Terms	7
Control groups	7
Randomization vs. random sampling	7
Between-subjects vs. within-subjects designs	8
Randomization	8
Between subjects designs	8
Within subjects designs (repeated measures)	9
Matched pairs designs	10
Example comparing between- and within-subjects designs	10
More types of experimental design	11
Factorial designs	11
Full factorial design	11
Fully-crossed vs. incomplete factorial designs	13
Balanced vs. unbalanced designs	13
Completely randomized design	13
Block designs	13
Randomized block designs	13
Latin square designs	14
Graeco-Latin square designs	15
Randomized Complete Block Design (RCBD ANOVA)	15
Split plot designs	17
Mixed design models	18
Pretest-posttest designs	18
Other forms of randomization	19
Lottery designs	19
Waiting list designs	19
Mandated control designs	20
Equivalent time series designs	20
Spatial separation designs	20
Mandated change/unknown solution designs	20
Tie-breaking designs	20
Indifference curve designs	20
New organizations designs	21
Quasi-Experimental Designs	21
Definition	21
One-group posttest-only design	21
Posttest-only design with nonequivalent comparison groups design	22
Posttest-only design with predicted higher-order interactions	22
One-group pretest-posttest design	23
Pretest-posttest design with before-after samples	23
Multiple-group pretest-posttest regression point displacement design	23
Two-group pretest-posttest design using an untreated control group	23
Double or multiple pretest designs	24
Four-group design with pretest-posttest and posttest-only groups	24
Nonequivalent dependent variables pretest-posttest design	24
Removed-treatment pretest-posttest design	25
Repeated-treatment design	25
Switching replications designs	25
Reversed-treatment pretest-posttest nonequivalent comparison groups design	26
Cohort designs with cyclical turnover	26
Interrupted time series designs	26
One-group time series regression-discontinuity design	26
Interrupted time series with a nonequivalent no-treatment comparison group	27
Interrupted time series with nonequivalent dependent variables	28
Interrupted time series with removed treatment	28
Interrupted time series with multiple replications	28
Interrupted time series with switching replications	29
Non-Experimental Designs	29
Definition	29
Examples	29
Frequently Asked Questions	30
Is a quasi-experimental design ever preferable to an experimental design?	30
Is qualitative research a type of quasi-experimental design?	30
How do I handle the problem of sample attrition in designs which involve observations at two or more time periods?	30
Bibliography	31
Pagecount: 34