Evaluation Decision Tree Tool
The evaluation decision tree tool is designed to empower you with essential skills and resources for evaluating your practice effectively. You’ll be introduced to key evaluation principles—including creative, transformative, and embedded approaches—that can shape your work meaningfully. We’ll guide you in finding practical templates, like Theory of Change, to support your unique evaluation plan, along with case studies and exemplars to inspire and inform your approach. You’ll also have access to valuable materials on access and participation planning, with clear direction on where to connect with relevant resources. Finally, you’ll strengthen your knowledge of evaluation methods to drive impactful and informed practices.
If you have a moment, we would love to hear whether this tool has been helpful for you here.
Instructions – go through the questions to find the right evaluation approach and tool for you
This has been developed in collaboration with ITL and APP.
Give your feedback here (need a Qualtrics)
Evaluation Guidance based on size and scale
Level 1 - Local (Course Level)
Definition:
Your change project will primarily impact your local level of the University e.g. team/programme level. You will be responsible for your own data collection and will require smaller levels of consultation.
Data Considerations
Data collection is personal. You are responsible for collecting data through channels already accessible to you. Dissemination is primarily internal. You may wish to share findings through relevant SharePoint sites and Newsletters.
Internal Example:
Clinical Data Science
The PG Cert in Clinical Data Science is a co-created course developed in partnership with public patient representatives, consultation with The Christie and NHS England. The course is aligned to NHS values, aiming to upskill the healthcare workforce with data science, statistics, machine learning capabilities and change management knowledge and skills. By planning the evaluation using a Theory of Change (ToC), the plan was broken down into three phases; short, medium and long term with separate outcomes for each, to see if the course had:
- developed foundational clinical data science knowledge, confidence and skills
- provided real world case studies (or exemplars) and authentic coding tasks
- personalised the learning enough to meet the needs of all students?
- given a taste of interdisciplinary / team science working?
- Encouraged all students to exercise agency and develop self-regulated learning skills?
- been an enjoyable experience that learners would recommend
- supported a broad mix of learners from across professional disciplines and levels
By phasing the outcomes, it was easier to see how each element contributed to the final impact which was to enable all learners to embark on a digital transformation project of their own. The evaluation is currently underway, with lessons learnt being collected which will be collated into a case study later in 2025
External Example:
Process and impact evaluation of the PGCert and MA Student Engagement in Higher Education programme
This process and impact evaluation example was created by Alan Donnelly and Liz Austen (2022) at Sheffield Hallam University. It is a ToC example that is clear and easy to follow.
How are you incorporating student voices?
Are students going to benefit from the intervention?
- You could look to student reps, external groups, listservs
Resources and Support:
Internal Evaluation drop-in sessions:
- UMITL3010 Evaluation Drop-in Support Session
- UMIT3009 Theory of Change Consultation Clinic
What's needed to run your evaluation?
No additional funding is required to run this evaluation
Level 2 - Semi-Organizational (School Level)
Definition:
You may begin to use data available from central BI reports and now require consideration of other change projects around you, to avoid potential attrition. You may require further levels of consultation e.g. >3 hours of consultation to plan methods etc.
Data Considerations
School level data. APP – planning dashboards, BI availability.
Consider if other projects are happening at this level
Dissemination at across the institution and maybe externally. Access and Success projects may share findings through the Access and Success Annual Impact Report.
Internal Example:
Manchester 10/10 Black Leadership Programme
The Manchester 10/10 Black Leadership programme (BPL) is a student success initiative which provides coaching for students of black heritage. This coaching aims to support participants to challenge mindsets which hinder success, increasing confidence in their studies and enabling them to take up leadership roles. Through their leadership roles, participants will be empowered to co-create improvements in student experience, helping the University to better understand how to support student of black heritage. It is the ambition of the BLP that these changes will result in an improved sense of belonging for students of black heritage and positively impact the degree awarding gap between white and black students.
External Example:
tbc
How are you incorporating student voices?
You can gather student voices as part of your evaluation design by:
-
Contacting the student perspective group via Student Communications via xxx in order to pay for student time using Amazon Vouchers
Resources and Support:
Internal Evaluation drop-in sessions:
External links
Evaluation collective - https://evaluationcollective.wordpress.com/
TASO - https://taso.org.uk/
Quality Assurance Agency - https://www.qaa.ac.uk/the-quality-code/2018/advice-and-guidance-18/monitoring-and-evaluation
OfS – Standard of evidence and evaluation https://www.officeforstudents.org.uk/for-providers/equality-of-opportunity/evaluation/standards-of-evidence-and-evaluation-self-assessment-tool/
What's needed to run your evaluation?
- Involves some funding to run the intervention to support.
- Involves other teams at School or Faculty level to support the intervention and evaluation, try to approach these teams early in the process to secure their support.
Level 3 - Institutional/Faculty Level
Definition:
You should consider having work streams or a task and finish group to manage the delivery and evaluation. Your project is of high relevance and importance for key UoM strategic objectives e.g. APP etc. Your evaluation may be informed by targets at an APR or APP level.
Data Considerations
APR/SEAP awareness – relation to wider change.
Full consultation AKA workstreams, T&F group.
Dissemination should be broad and likely beyond the scope of the institution. You may wish to publish your results in a blog or paper.
Internal Example:
Student Success Hub
The Student Success Hub (SSH) within the Faculty of Biology, Medicine, and Health comprises five workstreams, all seeking to address institutional drivers of differential attainment. The five workstreams are Accessing and Analysing Attainment Data, Embedding Inclusive Education, Developing Anti-Discriminatory Practice and Ensuring Equity of Placements. Through a multifaceted approach the SSH aims to make teaching and learning in FBMH more equitable and positively impact the degree awarding gap between white and black students and white and Asian students.
External Example:
Using learning analytics to prompt
student support interventions
‘Transforming Access and Student Outcomes in Higher Education (TASO) commissioned two English HE providers (HEPs) – Nottingham Trent University (NTU) and Sheffield Hallam University (SHU) – each to carry out a randomised controlled trial (RCT) of different interventions prompted by Learner Analytic (LA) systems. The trial at NTU tested established practice while that at SHU focused on a pilot intervention.
At each HEP, students on the trial were randomly assigned to one of two groups (Intervention 1 or Intervention 2) which determined which intervention they would receive if the LA system identified them as being at risk. In both groups, at-risk students received an email alert which contained details of available
support. For at-risk students in the Intervention 1 group, the email was automatically followed by a support phone call from a student adviser.’
How are you incorporating student voices?
As an institution-level project requiring resource, you could set up:
- Request a student partner for your initiative? - https://www.yoursay.manchester.ac.uk/partners-and-consultations/
Resources and Support:
Internal Evaluation drop-in sessions:
Evaluation collective - https://evaluationcollective.wordpress.com/
TASO - https://taso.org.uk/
Quality Assurance Agency - https://www.qaa.ac.uk/the-quality-code/2018/advice-and-guidance-18/monitoring-and-evaluationOfS
OfS – Standard of evidence and evaluation https://www.officeforstudents.org.uk/for-providers/equality-of-opportunity/evaluation/standards-of-evidence-and-evaluation-self-assessment-tool
What's needed to run your evaluation?
- Involves some funding to run the intervention to support
- Involves other teams at School, Faculty and Institution level to support the intervention and evaluation. Therefore, it is important to identify these as early as possible in order to get their support. For example, if you will be publishing the evidence externally and require ethics, you will need to factor in a long lead in time