Where significance deals with Type I errors (false positives), power deals with Type II errors (false negatives). In social science, typically the researcher wishes the significance level to be .05 or lower (less than 5% chance a relationship as strong or stronger than that observed would occur just by chance of sampling -that is, less than 5% chance of a Type I error) and the researcher wishes the power to be .80 or higher (less than 20% chance that a relationship found non-significant is actually significant; put another way, at least 80% chance of not making a Type II error)).
The larger the sample size, the easier it is for the researcher to achieve the .05 significance and .80 power cut-offs. One form of power analysis is directed at determining in advance of sampling the minimum needed sample size to achieve the needed level of power. Too small a sample will expose research findings to Type II errors due to insufficient power. Too large a sample incurs unnecessary expense for the research project.
Just as there are a large number of tests of significance (ex., chi-square significance, significance of a correlation coefficient, significance of a regression weight, etc.), so too power analysis will vary according to the significance test under consideration. In SPSS, the power analysis module is "SamplePower", which supports power analysis for tests of the significance of means and differences in means, proportions and differences in proportions, correlation, oneway and factorial analysis of variance (ANOVA), analysis of covariance (ANCOVA), regression and logistic regression, survival analysis, and equivalence tests for means and proportions. Below, the free, widely-used G*Power package for power analysis and sample size estimation is also presented.
The full content is now available from Statistical Associates Publishers. Click here.
Below is the unformatted table of contents.
Table of Contents Overview 5 Key Terms and Concepts 5 Purpose 5 Types of error 6 Type I Error 6 Type II Error 6 Factors affecting power 8 Desired power level 8 Desired significance level 8 Desired effect size to detect 8 Variance or variation of the dependent variable 9 Sample size 9 Examples using SPSS SamplePower 10 Power analysis for t-tests 10 Power analysis for multiple regression 13 Examples using G*Power 18 Overview 18 Using G*Power for the t-test example 18 Using G*Power for the multiple regression example 22 Assumptions 25 Random sampling 25 Post hoc power estimation 25 Use of confidence intervals instead of post-hoc power analysis 26 Post-hoc use of prior data for future power analyses 26 Acceptable use of post-hoc power analysis 26 Frequently Asked Questions 27 What tests does SPSS SamplePower support, and how does it work in general? 27 What is the SAS support for power analysis and sample size calculations? 27 What other power analysis tools are available on the Internet? 28 Russ Lenth's Java Applets for Power Analysis and Sample Size 28 DSS Researcher's Toolkit 29 RaoSoft Sample Size Calculator 30 Others 31 What is precision analysis? 31 Bibliography 33