#1 Define the objectives of your STEM outreach
What do you intend to achieve through your initiative? Outlining the change, you want to bring about (i.e., your STEM outreach outcomes) will highlight what evidence you might need to demonstrate the impact of your work. This is a key step to get you started in planning your evaluation approach.
Not sure on how to articulate your outcomes? Take a look at our Impact Framework, based on three elements that enable behavioural change: in order to make an informed decision as to whether an engineering career is right for them, young people need the capability, opportunity and motivation to do so. The Measures Bank offers a summary of Impact Framework outcomes linked to these elements. Our aim is to support the engineering community and consider how to measure the collective and long-term impact of STEM outreach on young people’s educational and career choices.
#2 Focus on what you need to evaluate
Don’t try to collect data on everything about your STEM outreach. With your outcomes in mind, stay focused on the priorities for your STEM event or programme and clarify your evaluation aims. This will help you consider exactly what you need to evaluate, guide the approach that would be best suited for your context and assign the adequate resources and realistic timelines.
The Measures Bank includes filters for you to view different types of measures based on what aspect of your programme you’re interested in evaluating. You will find example questions to measure how successful your initiative is in achieving your outcomes (impact measures), to collect feedback on the STEM outreach delivery (process measures) or assess how experiences and impact vary among participants by their characteristics (context measures).
#3 Decide who your participants will be
Once you have a clear idea of your STEM outreach outcomes, and the objectives of your evaluation it will be easier to think through who you will need to survey to collect the information you need.
Currently, the Measures Bank offers example questions for surveys aimed at young people, teachers, and parents. Through the tabs of this interactive resource, you’ll be able to easily navigate to relevant examples for surveys aimed at your target audience. For additional guidance on designing, planning and conducting evaluations with young people, beyond using surveys, check out our evaluation shared practice guide.
#4 Keep it simple
When designing your surveys only keep the questions that are strictly necessary. Here you can truly leverage the Measures Bank as a resource to view examples of measures that map onto the Impact Framework. We have taken the time to review over 250 survey questions and categorise them to make it easier for you to find what you’re looking for. Each impact measure in this resource is tagged with a topic that maps onto the three key elements of the Impact Framework and their related outcomes.
Our guide on evaluating using surveys, offers additional shared practice guidance on designing effective surveys for evaluating your STEM programme or activity.
#5 Involve key stakeholders in the process
You don’t need to do this alone! As you plan your STEM outreach and evaluation approach, consider involving stakeholders, such as funders, programme staff, volunteers, teachers, young people, or evaluation experts to help you design a tailored approach for your STEM outreach. Not only can it be helpful to leverage others’ expertise and ensure a successful evaluation, but this process can also foster buy-in for your STEM outreach.
Just as involving stakeholders in the development and planning of STEM outreach and evaluation can be beneficial, we’d like to learn from others across the STEM outreach community to build on the measures we currently have. Our resource is based on survey questions from EngineeringUK’s research and evaluation activities, including our Engineering Brand Monitor and evaluations we run with teachers and young people.
Our plan is to update the Measures Bank periodically as we learn from our own work as well as from feedback and contributions from the STEM outreach community. We aim to build on this tool and collectively improve the way we evaluate the impact of our STEM outreach efforts. So if you have any feedback on the resource, or would like to share any measures that have been working well for you, please do get in touch here.
