POWER ANALYSIS

Overview

Where significance deals with Type I errors (false positives), power deals with Type II errors (false negatives). In social science, typically the researcher wishes the significance level to be .05 or lower (less than 5% chance a relationship as strong or stronger than that observed would occur just by chance of sampling -that is, less than 5% chance of a Type I error) and the researcher wishes the power to be .80 or higher (less than 20% chance that a relationship found non-significant is actually significant; put another way, at least 80% chance of not making a Type II error)).

The larger the sample size, the easier it is for the researcher to achieve the .05 significance and .80 power cut-offs. One form of power analysis is directed at determining in advance of sampling the minimum needed sample size to achieve the needed level of power. Too small a sample will expose research findings to Type II errors due to insufficient power. Too large a sample incurs unnecessary expense for the research project.

Just as there are a large number of tests of significance (ex., chi-square significance, significance of a correlation coefficient, significance of a regression weight, etc.), so too power analysis will vary according to the significance test under consideration. In SPSS, the power analysis module is "SamplePower", which supports power analysis for tests of the significance of means and differences in means, proportions and differences in proportions, correlation, oneway and factorial analysis of variance (ANOVA), analysis of covariance (ANCOVA), regression and logistic regression, survival analysis, and equivalence tests for means and proportions. Below, the free, widely-used G*Power package for power analysis and sample size estimation is also presented.

The full content is now available from Statistical Associates Publishers. Click here.

Below is the unformatted table of contents.

Table of Contents

Overview	5
Key Terms and Concepts	5
Purpose	5
Types of error	6
Type I Error	6
Type II Error	6
Factors affecting power	8
Desired power level	8
Desired significance level	8
Desired effect size to detect	8
Variance or variation of the dependent variable	9
Sample size	9
Examples using SPSS SamplePower	10
Power analysis for t-tests	10
Power analysis for multiple regression	13
Examples using G*Power	18
Overview	18
Using G*Power for the t-test example	18
Using G*Power for the multiple regression example	22
Assumptions	25
Random sampling	25
Post hoc power estimation	25
Use of confidence intervals instead of post-hoc power analysis	26
Post-hoc use of prior data for future power analyses	26
Acceptable use of post-hoc power analysis	26
Frequently Asked Questions	27
What tests does SPSS SamplePower support, and how does it work in general?	27
What is the SAS support for power analysis and sample size calculations?	27
What other power analysis tools are available on the Internet?	28
Russ Lenth's Java Applets for Power Analysis and Sample Size	28
DSS Researcher's Toolkit	29
RaoSoft Sample Size Calculator	30
Others	31
What is precision analysis?	31
Bibliography	33