Power Analysis

Fast turnaround
even on larger
projects

Experience with even the most complex analysis

Expertise with the requirements of all major online universities

Perhaps the most frequently asked question that we receive from statistical consulting clients is, “What sample size do I need?” The answer to this question is influenced by a number of factors, including the purpose of your study, population size, the risk of selecting a “bad” sample, and the allowable sampling error.

In order to best estimate the sample size that you’d require or to validate that the sample you’ve planned is sufficient, you need to conduct a power analysis for your study. There are actually two kinds of power analysis: a priori, which helps you determine the minimum number of participants needed to justify your study, and post hoc, which tells you what statistical power is associated with a given sample size.

The a priori power analysis is typically the kind required for dissertation research, to ensure you have a clear target number of participants before you begin recruitment and data collection. We can absolutely complete your power analysis using G*Power, which is the most commonly used program for this work, in part because it provides some standard inputs as defaults–namely, effect size, significance level, and statistical power.

There are three major considerations when doing a power analysis for sample size determination, and our statisticians will consider each point carefully while completing your power analysis:

• The general approach to determining sample size assumes that a simple random sample is the sampling design. More complex designs, e.g., stratified random samples, must take into account the variances of subpopulations, strata, or clusters before an estimate of the variability in the population as a whole can be made.
• The sample size should be appropriate for the statistical analysis that is planned. If descriptive statistics are to be used, for example, then any reasonable sample size will suffice. On the other hand, a good size sample, approximately n=150+, is usually needed for multiple regression, analysis of covariance, or log-linear analysis, which might be performed for more rigorous state impact evaluations. In addition, an adjustment in the sample size may be needed to accommodate a comparative analysis of subgroups (such as an evaluation of program participants with nonparticipants). For all of these cases, before conducting a power analysis, you must have identified all necessary tests to analyze your data–and if you are still finalizing your statistical analysis, we can absolutely assist.
• Finally, the sample size formulas provide the number of responses that need to be obtained. Many researchers commonly add 25%+ to the planned sample size to compensate for persons that the researcher must remove from the sample for some reason. The number of surveys or questionnaires also can be substantially larger than the number required, based on the assumed response rate.

We also can conduct power analyses on more complex psychometric studies using structural equation modeling. For this type of study, we would need details on how you plan to set up the SEM (latent variables, their indicators, their correlations, etc.). We can then conduct your power analysis based on the methods described in McCallum, Browne, and Sugawara (1996, also cited in Klein’s SEM text, Principles and Practices of Structural Equation Modeling). This method involves performing a custom SAS routine for calculating power and sample size needed in order to accept or reject the null hypothesis that the model is a good fit of the data, based on the Root Mean Square Error of Approximation (RMSEA) metric.

Many of our statistical consulting clients seeking a power analysis are often on a short turnaround to finalize their methods section for chair or reviewer approval. If you require a power analysis at this stage of your project, or any other preliminary statistical consulting, we can typically provide this for you within as little as 1-2 days.