Measuring Impact

Collecting Data

Analysis of Data

Data analysis can seem a bit daunting, but there are a few things you can do to simplify the process:

Look at the big picture. What are the overall trends in the data?
Seek patterns and relationships in the data. Are there themes that appear? Are there competing themes? Do either of the competing themes present in a repeatable fashion?
Know that data analysis will have different steps depending on whether you are using quantitative or qualitative data.

When using quantitative data, you will likely use statistical methods to analyze your data. This analysis can help you to understand the project’s impact by looking at changes in project outcomes over time.

The following three factors must be taken into consideration when choosing an acceptable statistical method:

the goals and purposes of both the study of the project and the project itself
the type and distribution of the data used
whether the observations are paired or independent

All statistical techniques used to compare means are known as parametric techniques. Nonparametric techniques are used to compare measures other than means, such as median, mean ranks, and proportions.

Often project impact measurement benefits from the use of analysis software to evaluate the quantitative data gathered. Remember to take care in choosing this tool. Some points that should be checked include:

Is the software regularly maintained?
When was it last updated?
What are the noted reviews of the software?
Does the software have transparent hosting?
Who is providing the software?
Does the software respect user privacy?
- Does it protect personally identifiable information?
- Does it protect from information being gathered by artificial intelligence?
- Is any such data shared with a third party?
What are the limitations of the software?
- For instance, does it rely on one browser type, such as Chromium, for any automated data pulls?
- Are any other other information gaps are present in the software?
Are there any apparent cultural biases with the software?
What data outputs from the software need to be verified by a human?

When using qualitative data, you will likely use thematic analysis to analyze your data. This process involves looking for patterns and themes in the participants’ stories. This can help you to understand how the project has affected their lives. Thematic analysis focuses on identifying patterns, or themes, within the data.

Steps involved in thematic analysis are:

Familiarization
Coding the data
Highlighting key themes
Re-evaluating the themes
Defining and naming the themes
Creating a report to share your observations.

Don’t forget:

Quantitative analysis looks at changes in project outcomes over time and can be used to measure project impact.
Qualitative analysis looks for patterns and themes in participants’ stories and can be used to understand how the project has affected their lives.

Combining both quantitative and qualitative data helps to create a more complete picture of project impact.

Communicate Your Findings

Your project impact data gathering and analysis lack purpose if they are not shared with others. There are many options for reporting your conclusions. Whatever method you choose, make sure you communicate your findings in a way that is clear and concise. Your audience should be able to understand the findings and see the project’s impact.

When communicating your findings, it is important to:

Present the data in a visually appealing way.
Use charts, graphs, and tables to present quantitative data.
Use quotes and stories to present qualitative data.
Make sure your writing clearly conveys the data.
Focus on the project’s impacts rather than project outcomes.

When running a long-term project, it is important to plan for cycles of evaluation. Your project will change over time as your objectives and goals change. Reevaluating your project impact goals should happen each time your team updates its business, strategic or community goals.

In summary, project impact measurement is the process of understanding how your project has affected the lives of those it was intended to help. It is important to collect data on project outcomes and use both quantitative and qualitative analysis to understand the project’s impact. And remember to share your results with project stakeholders and beneficiaries!

I want to thank Sarah Womer for sharing her deep knowledge of data gathering and analysis.