Program Evaluation and Policy Analysis

At Precision, we have the expertise to help you evaluate your government, non-profit, or company program for process, performance, and impact. Whether you want to determine how best to measure outputs and outcomes, or if you need to identify the reasons for current levels of performance and impact, our team can design your evaluation to pinpoint what will best help your program or organization to exceed all expectations.

Our program evaluation consultants have extensive experience designing and implementing both formative (process) and summative (outcome) evaluations at the project, program, portfolio, and organization levels of analysis, for companies, government agencies, and non-profits in the US and abroad. We can also help you determine the extent to which an evaluation of your project, program, portfolio, or organization is feasible or necessary.

We have substantial experience designing and implementing program evaluations for non-profits and government agencies implementing projects and programs aimed at addressing complex issues including homelessness, addiction, hunger, human trafficking, and mental illness. We’ve also assisted in design and implementation of program and portfolio evaluations for industry, government, and universities in the areas of terrorism surveillance, smart vehicles, materials science and engineering, and freeform optics.

Our program evaluation support includes any (or all) of the following aspects of your upcoming or ongoing evaluation:

• Logic modeling
• Feasibility study
• Evaluation design
• Evaluation analytics
• Research centers and team science evaluation
• Student mentoring

Logic Modeling

First and foremost, any evaluation requires a valid logic model. At Precision, we will pair you with one of our logic model consultants to develop a thorough understanding of the inputs, processes, outputs, shorter-run outcomes, and longer-run impacts for your project, program, or organization.

We go beyond a simplistic “if-then” rationale for developing logic models, as these statements and their underlying assumptions typically don’t stand up to empirical scrutiny. In contrast, we dig deep into the relevant academic, empirical literature, examining in particular past evaluations similar to what you want to design and implement. Our evaluation consultants will then develop an evidence-based logic model for your project, program, or organization.

Feasibility Study

Is your project, program, portfolio, or organization ready for evaluation? If so, should you evaluate processes and outputs, outcomes and impacts, or some combination thereof? Is there enough participant and stakeholder willingness to ensure that the required allocation of finances, time, energy, and other resources will occur to ensure a valid evaluation of your project, program, portfolio, or organization?

Precision’s evaluation team can help you determine the extent to which your project, program, or organization is prepared for the investment of resources it takes to design and implement a formative or summative evaluation. We can assist you with an environmental scan of stakeholders, as well as with a systematic investigation of the extant and new data sources that will be required for your evaluation.

Evaluation Design

Our evaluation consultants can design your evaluation to address the questions that matter most to your project’s efficiency and effectiveness. Whether you’re focused on a systematic investigation of processes and implementation, or on the quantitative modeling of outcomes and socioeconomic impacts, we will ensure your evaluation design meets the criteria and expectations of internal and external stakeholders alike.

For summative or outcome evaluations, Precision will design a unique evaluation for your project, which isolates and identifies the extent of impact for improvements in unit performance—as well as for longer-run social and economic impacts. We’ll take into account the unique context and internal characteristics of your project to ensure you’re credited for positive net outcomes. We will design your evaluation to isolate net effects or causality to the extent that your data allow, and also, of course, recognize and address with statistical controls any threats to internal validity.

If it’s still too early to evaluate your project, program, or organization for outcomes and impacts, our consultants can design a formative or process evaluation study to diagnose any operational factors that may in time hinder effectiveness, efficiency, and ultimate survival. We’ll address internal processes, informed by our subject matter experts in industrial organization, organizational psychology, organizational economics, and leadership and management—as well as external stakeholders, such as your program’s target audience or clientele, financial and political stakeholders, and the broader public.

Evaluation Analytics

Precision’s modus operandi is rigorous analysis. We have over 100 analysts and methodologists who are ready to assist you with quantitative evaluation analytics as well as with qualitative evaluation analytics of stakeholder and process content. Our evaluation team will reveal the underlying relationships and structure of your quantitative data and qualitative content, respectively, to inform decision-making regarding your project, program, or organization.

In terms of your quantitative outcome and impact data analysis, we know how to properly structure, code, and analyze your dataset for your outcome evaluation. Our statisticians have extensive experience in dataset development and analysis for pretest-posttest, interrupted time series, and difference-in-differences evaluation designs, as well as for more advanced designs when the data allow—such as regression discontinuity analysis and even natural experiments. We know when to run what statistical test, and which multiple regression or analysis of variance technique to employ to ensure that your evaluation results are rigorous, replicable, and valid. Let us help you get the most out of your outcome and impact data, so you can get the most out of your program or organization.

For those seeking qualitative content analysis of your interviews, focus groups, and program documentation, we are experts at revealing the underlying structure of qualitative content for process evaluation. Using your interview and focus group transcripts, program or organization documentation, public reviews of your non-profit, government agency, or firm from Google and Yelp, and the transcripts from your program participant and stakeholder interviews, our consultants will make sense out of the seemingly overwhelming amount of information you have at your disposal. Let us help you to identify strengths and weaknesses so that you can keep doing what’s working, and change what’s not.

Research Center and Team Science Evaluation

Our evaluation team is focused on the strategic management and evaluation of academic and government research and development investments, including but not limited to collaborative, multi-disciplinary, multi-sector university research centers, such as those sponsored by the US National Science Foundation, US National Institutes of Health, and comparable collaborative research centers initiatives and programs.

Our philosophy is that there is not one best way to incent, structure, manage, and evaluate collaborative research centers involving universities, industry, and government. In other words, what works for one research center may not work for yours, and what worked in the past for your team science program may not work moving forward. At Precision, we design customized evaluations to help you to demonstrate the public value and other socioeconomic returns on investment for your collaborative research center—or to help you to develop improved prospects for these outcomes.

Our clients include both sponsors and performers of science and engineering research and development in the public interest. Our research center evaluation services include the following:

• Strategic consulting for grant proposals and active grants—implementation and evaluation strategies for multi-institutional and multi-disciplinary projects and programs.
• Evaluation design and implementation—formative and summative, using pre- and quasi-experimental designs and advanced quantitative and qualitative analytic techniques.
• Broader impacts assessment—strategy development to demonstrate socioeconomic changes resulting from public research investments.
• Decision tool development—informed decision-making when vetting proposals and evaluating the performance of research and development investments in the public interest.
• Portfolio analysis—for academic units (e.g., colleges, academic programs, research centers) and research sponsors (e.g., agencies, agency programs, multi-agency programs).
• Feasibility study—real-time and ex post facto, to determine the need and potential for project and program evaluation and broader impacts assessment.

Student mentoring

If you’re a graduate student in a public policy, public administration, or public health program, chances are you have to design and implement a program evaluation study. Precision’s team includes former program evaluation, policy modeling, and quantitative and qualitative evaluation methods professors to mentor you with these requirements.

• Worried about your logic model? At Precision, we have program evaluators with decades of experience who can help you to connect the dots for your focal program, policy, non-profit, or law. They can not only assist you in populating your inputs, processes, outputs, and outcomes and impacts columns, but also with informing your logic model with the assumptions that underlay the “if-then” connection between columns with an extensive and systematic review of the extant evaluation and academic literatures pertinent to your program’s or policies theory of change.
• Worried about identifying valid and reliable data? Our evaluation team has established expertise with developing new data and identifying and using extant data for public program and policy evaluation and analysis.
• Worried about internal validity? Our statistical consultants can help you not only to design a quasi-experimental evaluation for the program, policy, law, or non-profit of your choosing, but also to understand you to understand the strengths and weaknesses of your design vis-à-vis history, maturation, testing, instrumentation, selection, and regression to the mean.
• Worried about selecting an appropriate non-equivalent comparison group? At Precision we can assist you with identifying what factors or variables matter (and which ones don’t) when it comes to identifying a valid comparison group. We can also perform the corresponding analyses to ensure that the group you identify and collect data for is a good fit for your quasi-experimental program evaluation.