Your evaluation questions answered!

Abby Sabey, Senior Lecturer at the University of the West of England (UWE) also delivers training with the West of England AHSN. Here she reflects on the recent delivery of the ‘Introduction to Service Evaluation’ course and answers some key questions…

I ran the ‘Introduction to Service Evaluation’ course with my expert evaluation colleague Kathy Pollard from UWE, in early December. On this one day course for health service providers and commissioners, we share experience of successful evaluations and facilitate group work that gives participants the opportunity to develop their own evaluation ideas. Like every other occasion I was bowled over by the commitment, thoughtfulness and hard work of everyone who came along to join us at CLAHRC West.

Earlier this year we refreshed the course and built in a small element about using the principles of logic modelling to develop an evaluation plan. Logic modelling helps you tell the story of your project in a diagram and a few simple words, helping you test the logic of what you plan to do and how this will make a difference. It worked its magic again this time and everyone took away a completed plan on which to base their evaluation.

The group who joined us included people working in public health, medical physics and commissioning; highlighting the wide range of people who are getting involved in evaluation. There was a great communal feeling in the room during the course. We give lots of opportunity for people to interact and discuss their projects as a way to test out and develop ideas and this time in particular, it was noticeable how much support there was within the group.

Towards the end participants showed how far they had come by presenting a three minute summary of their plan; great evidence of a productive day. We were delighted to get an almost perfect four out of four in the course evaluations; a great morale boost at the end of another year of training!

If any of this has given you an appetite for evaluation, feast on these top tips and resources:

1. What is evaluation and why is it important?

Evaluation is a way of learning what works, why and how…and why things don’t work. It leads to change and improvement; enhances quality; and shows accountability and organisational learning. Evaluation also contributes to the evidence base. This short video explains more

2. I need to do an evaluation, where do I start?

The Evaluation Works toolkit! It provides a step by step approach to completing an evaluation through five steps of the evaluation cycle.

Only got five minutes? Watch this short video on the first step of the cycle and check out the quick start guide

Got half an hour? Watch the full collection of videos that make up key components of the ‘Getting Started with Service Evaluation’ course here www.nhsevaluationtoolkit.net/resources/training

Got more time? Work through the toolkit step by step or come along to the next training workshop.

3. Where can I find out more?

  • CLAHRC West offer a variety of training courses. Keep an eye out for the next ‘Introduction to Service Evaluation’ on their website
  • Learn from the experience of others. There are several case studies on the West of England AHSN website demonstrating the positive impact of service evaluation
  • You could also sign up to the evaluation online network; a virtual peer to peer support group for all things evaluation related
  • And of course, the Evaluation Works toolkit is full of advice and resources

Goodbye 2017, hello 2018!

Drawing on lessons from our Patient Safety team’s recent away day, Nathalie Delaney shares a few creative and collaborative ideas to review what you’ve achieved as a team in the past year and how this might inform your plans for the coming year.

At the end of the year, it can be a good time to reflect on the achievements and learning of the past year and make intentional plans for a successful year ahead.

The West of England AHSN’s Patient Safety team met at the end of 2017 to do this together, and I thought I’d share our process here as you may want to reflect on this yourself or with your own team.

Goodbye 2017

As an icebreaker, we looked at the strengths of our team using the 16 personality types. Who’s in your team? What are your strengths? Then we looked back to celebrate the past year and asked ourselves:

  • What were our successes and achievements in the past 12 months?
  • What have we learned about ourselves and our members this year?
  • What were our biggest lessons this year?
  • Was there anything that surprised us?

If you are doing this as an individual, you may want to start by reviewing your calendar for the past year to identify key milestones and achievements. Alternatively you could create a shared timeline as a team. The Sign Up to Safety Kitchen Table Scribble Sheet has some other prompts that may help you do a freeform download of what’s on your mind.

Based on this we made a list of: what might we change or maintain in our work in 2018?

Looking ahead

We then had a presentation from our Managing Director, Deborah Evans, and Chief Operating Officer, Tasha Swinscoe, on the ‘big picture’ for the year ahead, both nationally and locally. From these, we identified what opportunities and challenges might arise in the year ahead?

Each member of the team was asked to nominate their theme song before the workshop. We held a short quiz to try to guess which song belongs to each team member, based on 20-second snippets from Spotify. As a team, we were surprisingly bad at this and it was a good way to find out something new about colleagues we work with. In case you’re interested, mine was ELO’s Mr Blue Sky.

Hello 2018!

The afternoon was spent creating a shared vision for 2018. We split this into aspects about how we, the team worked, and also our projects.

For our team we asked:

  • What should we stop this year?
  • What should we continue this year?
  • What should we start this year?

You can download a template to use with your team or consider the questions individually.

After a quick cake break, we discussed our projects and objectives, using the MoSCoW framework to identify our key aspirations and the impact we wanted to see:

  • Must do
  • Should do
  • Could do
  • Won’t do.

Our next steps are to identify what results we want to see and break these down into our large goals and smaller steps as driver diagrams using Quality Improvement methodology. As a team, we are moving to using Verto to manage our programme in the New Year. If you prefer paper to digital, you can print out our handy calendar for 2018 and add in your own goals and actions.

Over to you…

How can we in the West of England AHSN help you reach your goals in 2018? There are plenty of opportunities for support whatever your aspirations.

If your goal is to learn more about Quality Improvement, why not sign up to study the Quality Improvement in Healthcare MOOC starting in February 2018. This is a free six week course delivered online. You can also sign up for a free account for LifeQI which is a web-based platform designed to assist frontline staff running Quality and Safety improvement. The LifeQI system has recently been majorly upgraded and is free to access for West of England AHSN members.

Opportunities to join the Q community, an initiative connecting people with improvement expertise across the UK will re-open in 2018. Register your interest to be notified when applications are open.

If you are working in primary care, recruitment will open soon for Cohort Three of our successful primary care collaborative. Find out more and register your interest.

If your goal is to share learning from an innovative, measurable solution to a patient safety issue then why not enter the National Patient Safety and Quality Improvement Poster competition. Enter online by 16 March 2018. Short-listed entries will be invited to present a poster at the Bristol Patient Safety Conference on 16 May 2018.

If you are working in business check out our business support video which includes an introduction to our work with businesses and innovators and sign up for our newsletters including our funding finder which provides regular up-to-date funding opportunities for health innovators. This includes opportunities from Innovate UK, Department of Business, Innovation & Skills, Horizon 2020, The National Institute for Health Research and NHS England.

If you want to meet and share ideas with others who share your passion about improving and innovating healthcare, then you need to sign up for hyvr, our new social network for citizens, innovators, healthcare professionals – and well anyone really!

And if you would like to know more about evidence and evaluation or involving public contributors we have a range of toolkits to help you with practical steps in your day-to-day work. NIHR CLAHRC West provide a range of excellent training courses aimed at developing of skills in understanding, using and producing evidence for the health, public health and commissioning workforce, and patients and members of the public. You can also join our evaluation online network, a virtual peer to peer support group for all things evaluation related.

Evidence Works – an evidence-informed commissioning toolkit

Evaluation Works – a service evaluation toolkit.

Working Together – A toolkit for health professionals on how to involve the public

All the best wishes for a fantastic 2018!

10 evidence-based ways the GP Clinical Evidence Fellows are making an impact

Ceilidh Jackson – Baker, Project Support Officer with the commissioning evidence-informed care team celebrates the triumphs of our GP Clinical Evidence Fellows…

In June 2017 I helped to organise an event to showcase the achievements of our GP Clinical Evidence Fellows. My team (commissioning evidence-informed care) support the ten Fellows to learn how to access and use evidence and then work with each of our seven clinical commissioning groups (CCGs) to embed this best practice into commissioning decision-making.

What really stood out for me at the showcase was the level of impact the Clinical Evidence Fellows are already achieving across a whole range of different areas. The feedback from attendees showed that CCGs also recognise their value.

Attending the GP Clinical Evidence Fellows’ event on the 13 June 2017 was really inspirational. The short presentations each GP Fellow gave highlighted the personal and professional journeys they had been on and how their knowledge of the commissioning world had increased. With clinical commissioning groups looking for evidence to inform the commissioning decisions, Clinical Evidence Fellows have a lot to offer.

Becca Robinson, Service Improvement Lead – Bristol Clinical Commissioning Group

Here, in no particular order, are my top ten ways our Clinical Evidence Fellows are influencing commissioning in the West of England:

  1. Improving interventions for frail elderly people

Dr Bisola Ezobi’s work looking into evidence on community-based interventions for frail elderly people for Bath & North East Somerset CCG has resulted in Comprehensive Geriatric Assessments being implemented in the community.

  1. Evidence reviews for clinical policies

Dr Catherine Bennett’s evidence searches relating to clinical policies have been used to guide the formal policy review and redesign at Wiltshire CCG to ensure that evidence underlies their decision-making.

We need to continue to support the Fellows and embed them into CCG life!” Wiltshire CCG representative

  1. Minor procedures demand management

Swindon CCG have moved most minor surgical procedures to the same evidence-based criteria as Wiltshire CCG as a result of Dr Francis Campbell’s work, thus reducing inequality of access to these services to the local populations.

  1. Near-Patient CRP testing

Dr Caroline Ward implemented a pilot study of near patient C-reactive protein (CRP) testing with the Urgent Care Unit in Swindon CCG. This is a simple pinprick blood test taken in the consultation which gives results in minutes as to whether a patient has an infection that needs antibiotics. Early results indicate that this has reduced unnecessary antibiotic prescribing, which is good for antimicrobial resistance – and therefore patient care and CCG prescribing budgets.

  1. Expert Patient Programme

Dr Vanessa Dane’s review of the available evidence for peer led self-management programmes was presented to the Gloucestershire CCG clinical programme teams who used the findings to inform development of the Expert Patient Programme.

The GP Clinical Evidence Fellowship is “a great initiative and I will continue to support and endorse the importance of evidence-based commissioning.” Gloucestershire CCG representative

  1. Supporting improvement of the COPD pathway

Dr Farida Ahmad has helped to increase awareness across Bristol, North Somerset and South Gloucestershire STP of the evidence that pulmonary rehab can make a positive difference for people with Chronic Obstructive Pulmonary Disease (COPD).

For a relatively small amount of time and money a huge amount has been achieved professionally and personally and in CCG benefit.” Bristol CCG representative

  1. Reviewing the use of digital apps in the treatment of depression

Dr Ed Mann’s review of the use of digital apps in the treatment of depression saved North Somerset CCG money through not investing in iCBT due to there being no strong evidence supporting its effectiveness.

Dr Mann’s evidence review “looked at specific models (and the evidence behind them) currently available on the market. It also pulled in the current commissioning context and demonstrated a knowledge of the current pathway and therefore opportunities to transform. It then weighed up the pros/cons.” North Somerset CCG representative

  1. Informing service change through effective use of RightCare data

RightCare is a programme for identifying unwarranted variation in health across CCGs. The RightCare data reports produced by Dr Charlie Kenward have been shared with various teams across North Somerset CCG and are informing service change. This has included transformation teams discussing the need to develop an evidence-based plan for relieving pressure on A&E.

  1. Reviewing injection therapy for treating low back pain

Dr Nick Snelling’s evidence review on treating low back pain with injection therapy has resulted in facet joint injections only being available by exception as the evidence suggests insufficient clinical effectiveness for offering these routinely. This means anticipated savings across Bristol, North Somerset and South Gloucestershire STP of approximately £100,000 per year, and alternative pathways to manage patients’ pain in the long-term.

  1. Suicide prevention in the LGBTQ community

Dr Phil Simons completed an evidence review on suicide prevention strategies for the LGBTQ community as part of the South Gloucestershire Suicide Prevention Strategy and made various recommendations for improving support and training, particularly in schools, which the partnership group are looking to implement.

Phil’s work has reinforced the LGBT community as one of our local priority groups for reducing inequalities… His review has helped shape what this might look like in practice, influenced strategy and specifically plans to provide more LGBTQ support in schools.” Public Health Programme Lead (Mental Health & Emotional Well Being) for South Gloucestershire Council

Find out more

the opportunity to network and discuss our roles provided a valuable opportunity for inter professional knowledge exchange…The morning was informative, well organised and personally motivating for me and was useful not only in my new role with APCRC, but also for clinical and teaching purposes. In conclusion, a great morning and time very well spent!

Dr Anne Johnson, Senior Lecturer/Researcher in Residence Fellow – Avon Primary Care Research Collaborative

Evidence Live 2017 – the Glastonbury of evidence-based medicine

Our Primary Care Programme Lead, Sian Jones shares her thoughts on this year’s Evidence Live…

20 June 2017 was one of the hottest evenings since the summer of 1976 to be heading to Oxford for the evidence nerd-fest that is Evidence Live. This two day international conference brings together the great and the good of evidence-based medicine (EBM), geeks and gawping groupies, as well as a lot of us who want to expand our knowledge and networks, or share what we’re doing.

Oxford University’s Centre for Evidence-Based Medicine (CEBM) and the British Medical Journal (BMJ) jointly run this event every year. Ben Goldacre of ‘Bad Pharma’ and ‘Bad Science’ fame, refers to the conference as “the Glastonbury of EBM”, where “everyone who is anyone passes through”.

Bristol on tour

This year I attended just one day to support Bristol colleagues presenting from CLAHRC West and UWE. We gathered in the stylish Blavatnik School for Government where the conference started with an opening address from Fiona Godlee, BMJ Editor-in-Chief.

To give a sense of day one, the plenaries covered:

  • Fergal O’Regan from the European Ombudsman on transparency of clinical trials data from the viewpoint of the European Medicines Agency.
  • Doug Altman; Oxford Professor of Statistics in Research, on the scandal of poor medical research. He explained that nothing much has changed over more than 20 years and says “ignorance of research methods is no excuse, if you can’t do it well, don’t do it”.
  • A thought provoking presentation from Mary Dixon-Woods; Cambridge Professor of Health Services Research, on improving evidence for improving healthcare, where many examples were presented to show how the evidence that quality improvement (QI) improves quality is not robust and evaluation is lacking. She shared the good news that Cambridge University has been awarded a grant from The Health Foundation to set up an Improvement Research Institute to strengthen the evidence base for improvement and she will be leading this.

Evidence-based medicine versus evidence-informed commissioning

A key focus of the conference was a consultation on the EBM Manifesto for Better Healthcare. This has been developed with the aim of finding solutions for better healthcare evidence, in response to the 2014 BMJ paper by Trisha Greenhalgh et al ‘Evidence based medicine: a movement in crisis’. The Manifesto is concerned with the increasing volume of evidence that is of variable quality and how this impacts decision making.

Simply put, bad (untrustworthy) evidence results in poor decisions. This clearly has implications for patient care. So the EBM Manifesto hopes to fix the problems in EBM asking for ideas from stakeholders via the ongoing consultation.

EBM has been around for over 20 years and has been adopted as an approach to support clinical decision making. I was reflecting on the view of the EBM leaders that EBM is in crisis, and thinking about commissioning – the area of healthcare we are working to influence at the West of England AHSN.

Commissioners are responsible for two-thirds of the NHS budget, yet the way they make decisions on how to allocate this has not been formally studied and there has not been an approach like EBM devised to support them. We also don’t suffer from too much research, whereas in EBM – the reverse is true. The Health and Social Care Act 2012 makes reference to the need for greater participation by Clinical Commissioning Groups (CCGs) in research, but before this there has been little requirement on commissioning organisations to apply research evidence.

Commissioners make decisions that affect populations: it’s a big deal! So it feels like commissioning is in an even worse place than EBM. It is not an issue of not wanting to use research, but it’s not always easy.

Our very own manifesto

Despite the heat, Evidence Live 2017 was a less nervy experience than 2016 when I co-presented a workshop on evidence-informed commissioning with Alison Turner from Midlands and Lancashire Commissioning Support Unit. The workshop nurtured an idea for a manifesto (yes another one!) for commissioning, setting out ten steps towards an ideal world of evidence-informed commissioning.

This has been blogged about here:

Our manifesto represents a call to action and we would encourage those who have an opportunity to influence commissioning decisions to use it as a set of aspirational standards.