Measurement for improvement does not have to be complicated. Tracking a few measures over time and presenting the information well is fundamental to developing a change that works well and can be spread.
Measurement can show us a number of important pieces of information:
- how well our current process is performing
- whether we have reached an aim
- how much variation is in our data/process
- whether the tests of change have resulted in improvement
- whether a change has been sustained.
Reasons for measurement
There are three main reasons why we measure:
- judgement, and
Understanding what you are measuring and why is vital as it determines how you approach the measurement process
In the health service we are more familiar and comfortable with measurement for research on a large scale with a fixed hypothesis or question; or for those in more strategic roles , with measurement for judgement as a way of understanding a level of performance.
Measuring for improvement is different. The concept of sequential testing means that there needs to be willingness to frequently change the hypothesis (as you learn more with each test) and an acceptance of ‘just enough’ data, working with data and information that is ‘good enough’ rather than perfect. Measurement for improvement does not seek to prove or disprove whether clinical interventions work – it seeks to answer the question “how do we make it work here?”
Types of measures
The three types we use in improvement work are called outcome, process and balancing measures.
- Outcome measures reflect the impact on the patient and show the end result of your improvement work eg .rate of infection cases.
- Process measures reflect the way your systems and processes work to deliver the outcome you want. Examples would be % compliance with hand washing
- Balancing measures reflect what may be happening elsewhere in the system as a result of the change. This impact may be positive or negative. When presented with change, people can be heard to say things like, “If you change this, it will affect that.” Picking up on the ‘thats’ can lead to a useful balancing measure
Collecting data will be an important element of your project, and here are some tools that will help you analysis the data.
Small amounts of data can be collected regularly and complied into ‘run charts’, or ‘control charts’ to look at review the impact of a change over a period of time. For example:
Run Chart – Infection rate vs Target
Run charts or control charts focus on variation. There is an important distinction between these and snapshot audits:
- A run chart acts a bit like a camcorder, showing you every up and down.
- Snapshot audits are more like a camera, taking a picture of what things look like at just one point in time.
To show that things have improved you need to show the things that have changed, and that the change is not a one off. You must consider whether the change has been sustained. Run or control charts allow you to see if this has happened.
Statistical Process Control (SPC)
This approach helps you to understand the scale of a problem, gathering information and identifying possible causes. It examines the difference between:
- Natural variation – also known as common cause variation
- Controlled variation – also known as special cause variation.
Control charts are used that display boundaries for acceptable variation in a process. The data collected over time shows whether a process is within control limits in order to detect poor or deteriorating performance and target where improvements are needed.
- The NHS How to Guide on measurement for improvement
- Read more about SPC from the former NHS Institute for Innovation and Improvement
- Mike Davidge on measurement for improvement
- IHI: Building Skills in Data Collection and Understanding Variation
- Whiteboard: Static vs Dynamic Data
- Whiteboard: Run Chart 1
- Whiteboard: Run Chart 2
- Whiteboard: Control Charts 1
- Whiteboard: Control Charts 2