George Bramley, Senior Analyst, City-REDI / WM REDI and Council Member of the UK Evaluation Society writes about the benefits and importance of evaluation during the COVID-19 pandemic, and how we can use online software to make networking and information sharing more effective.
The demand for evaluation is increasing within the region and nationally. Not just for evaluation of existing programmes but also as part of business case development for new proposals to help the region pivot and adjust to the challenges created by COVID-19. This summer WM REDI has supported a number of bids to Treasury for additional funds through undertaking ex-ante evaluation or appraisal of benefits. We continue to support organisations within the region to develop their skills in economic analysis, monitoring and evaluation. There is likely to be increased emphasis on evaluation in the revised Green Book which sets out how business cases should be developed for public sector investment and my colleague Rebecca Riley is leading on a national group to develop and share good practice in business case development.
COVID-19 has been a spur to innovation at the national and regional level in how the UK Evaluation Society (UKES) and other organisations support professional development and knowledge exchange amongst those who commission, use or undertake evaluation studies. WM REDI has been involved in many of these innovations through my membership of the national events group and UKES Midlands Regional Network.
A key date for your diary is the Midlands Evaluation Showcase on 5th November which includes leading evaluators based in the Midlands who will be talking about the exciting evaluation work they are undertaking in the Midlands and challenges they have been grappling within the light of the current pandemic. Rather than the usual seminar or conference format of presentations followed by a Q&A session, you can access in advance Showcase Contributions which are short video blogs by a number of contributors. Michelle Hollier and Paul Woodcraft who are based at Winning Moves describing their approach to evaluating skills programmes in the context of a pandemic and Adam Knight-Markiegi (M·E·L Research) shares his experience from evaluating Making Connections a social prescribing in programme Walsall using Social Return on Investment (SROI) analysis. Matthew Terry, Cloud Chamber. Matthew Terry discusses the recent evaluation of Brushstrokes, a small community-based project in the West Midlands supporting refugees, asylum-seekers and other vulnerable migrants. Matthew Horton, University of Birmingham. Matthew Horton describes the Aimhigher programme and work being undertaken to evaluate its impact and effectiveness, showcasing the use of a quasi-experimental approach to provide a robust evaluation of a large-scale intervention programme, and illustrating how the resulting evidence is being used to assess and address learner needs and improve outcomes.
On the day you will have the opportunity to participate in two panel sessions in which you can present your questions to our regions evaluation stars. Confirmed panellists include Michelle Hollier, Director of Research and Evaluation, Winning Moves, Matthew Horton, Widening Participation Research Manager (NCOP and Access Agreement), University of Birmingham, Mark Hart, Deputy Director of the Enterprise Research Centre and Professor of Small Business and Entrepreneurship at Aston Business School, Adam Knight-Markiegi, Research Director, M·E·L Research, Anne Green, Professor of Regional Economic Development, City Region Economic and Development Institute (City REDI), University of Birmingham, Matthew Terry, Evaluation Specialist, Cloud Chamber and Lauren Amery, Business Intelligence Analyst, Birmingham Children’s Partnership.
While I love to get together at events and conferences to share experiences in person with others in evaluation, I do accept that the use of Zoom or other videoconference formats increases accessibility to fellow evaluators who may not have the time, travel or training budget to attend such events. Therefore innovations such as Evaluation Matters which is a new format being tested involving a series of lunchtime sessions during the week of 23rd November are a welcome development. Like the national annual conference, this is open to both members and non-members of the UK Evaluation Society and the timing of the sessions 12.30 to 2.00pm means colleagues from the Americas as well as Europe will be able to join.
A series of on-line training events have been developed on new approaches to evaluation. In addition, in partnership with CECAN, a series of on-line training events have been developed on new approaches to evaluation including process tracing, Bayesian updating and qualitative comparative analysis for assessing impact in more complex interventions as well as a set of well-attended webinars on topics including ‘Top Tips for Evaluators’ covering how to respond to COVID, ‘Evaluation in government – how to create Impact’, ‘Let’s digitalize’ and the launch of the revised Magenta Book by the government to the wider evaluation community.
My own personal innovation has been the piloting of an Online Journal Club which is very similar to taking part in a Book Club but can be organised in different ways suited to the material being discussed. Just like a book club, participants are asked to read a journal article in advance so they can discuss it with the author and ‘fellow club members’. The author starts with a short presentation in which they set out why they wrote the paper, key points and issues they like to explore with the group before opening up the discussion. Because of the deliberate decision to keep meetings small, it has been possible to allow participants to ask questions and explore the relevance to their own work. Bridget Dillon has also created an Evaluators Webinar format which takes place on a monthly basis and literally recreates discussions around the conference bar.
These innovative events are helping create social networking opportunities needed to foster the sense of belonging to the professional community of evaluators at a time when demand for evaluation is high, but an opportunity for in-person interaction is low.
This blog was written by George Bramley, Senior Analyst, City-REDI / WM REDI.
Disclaimer:
The views expressed in this analysis post are those of the authors and not necessarily those of City-REDI / WM REDI or the University of Birmingham.
To sign up to our blog mailing list, please click here.