Collecting and analysing data on quality of care

Clinical governance seeks not just to maintain the status quo, but to promote continual improvement. For this purpose, appropriate data have to be collected and analysed, then applied in benchmarking and QI.

Clinical audit

What is healthcare audit?

Healthcare audit is the collection and analysis of healthcare data for the purposes of assessing quality of care, benchmarking, and im­provement of performance. The term ‘clinical audit' is used to de­scribe this activity but it is important to appreciate that audits may look not only at clinical outcomes but also at other health outcomes, such as quality of life and user satisfaction.

Types of audit

Healthcare audit can take any of a number of forms such as criterion­based audit, normative audit, or case notes review.

Criterion-based audit is the type of audit commonly represented in the form of an audit cycle (or spiral) that includes standard set­ting, comparison of current practice with standard(s), implemen­tation of change, and closure of the loop by re-audit—see Figure 5.2). Criterion-based audit starts off with evidence-based criteria and agreement on expected levels of performance, thus establishing both ownership and robust foundation at the outset.

Normative audit involves collection of data, usually on large numbers of patients, to identify trends, health outcomes, com­plications, readmission rates, or similar indices. By definition it may not start with criteria (as defined previously) but its findings could be used to set up local criterion-based audits. Examples of normative national audits include the audit to assess patient out­comes and experiences of care for women with heavy menstrual bleeding in England and Wales (33) and the National Pregnancy in Diabetes audit (34). The longest running national audit in the United Kingdom, the Confidential Enquiries into Maternal Deaths, is also a normative audit. At a local level, normative audits may be used to populate maternity dashboards (e.g. rates of third-degree perineal tear) or to monitor readmission rates or responses to early warning scores.

Case-notes (charts) review is a time-honoured type of audit in medical practice. A case record (in some cases, more than one) is re­viewed and presented at a meeting, and judgements are made about the quality of care, based on implicit standards (i.e. standards de­fined by the personal experience and perspective of the assessors). From the purist’s point of view, criterion-based audit is the highest form of audit. Indeed it is only this type of audit that fits the defin­ition of ‘clinical audit’ endorsed by NICE: ‘a quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implemen­tation of change’.

As outlined in the earlier section on clinical effectiveness, there is a hierarchy by which research evidence, and recommendations flowing from the evidence, are graded. This hierarchy can be rep­resented as a pyramid, with random allocation trials at the top and case series and expert opinion at the bottom (the pyramid on the left in Figure 5.3). A similar pyramid can be devised for a healthcare audit, with criterion-based audit at the top and case-notes review at the bottom (the pyramid on the right in Figure 5.3). Criterion-based audit is, in hierarchical terms, audit’s equivalent of the randomized controlled trial.

Figure 5.3 also illustrates the overlap between audit and research— some normative audits may be undertaken or presented in the con­text of research. For example, the Birthplace study (35) falls into the overlap between the two triangles, being a normative audit but also an observational research study.

The practicalities and challenges of clinical audit

There would be no purpose served if clinical audit does not lead to improvement and sustenance of clinical standards. This should be borne in mind when selecting a subject for audit. The chosen subject (a clinical condition or a process of care) should be pertinent to local practice, and one for which data can be collected readily.

When the subject/topic has been selected, the next step is to set out criteria and standards. A criterion is an evidence-based statement that can be used to assess the appropriateness of current practice. In an audit of perioperative care in caesarean section, for example, one of the criteria could be ‘Women undergoing caesarean section should have thromboembolism prophylaxis’. The standard specifies

Figure 5.2 The clinical audit cycle.

Figure 5.3 The research and audit hierarchies.

the expected level of performance for the corresponding criterion. This will commonly be 100% but in some cases, could be lower—for example, when local practice is being compared with a regional or national benchmark that falls below 100%. The criteria are usually derived from evidence-based guidelines or research findings. Where no guidelines exist, standards can be agreed by consensus locally.

A data collection pro forma is designed and tested on a handful of cases, and a final version is agreed by the audit team. The pro forma may be in paper and/or electronic form. The pro forma should pro­tect the anonymity of patients and staff.

Data are then collected for a pre- agreed number of cases or period. The minimum number of cases required for criterion-based audit is usually modest in comparison with research where it is usually an advantage to have large numbers of cases. Descriptive statistics are usually adequate for audit but more complex statistical analysis may be required for larger audits.

Current practice is compared with the set standards and any defi­ciencies are identified, as well as possible explanations for the short­fall. A plan of action is devised to change/improve current practice. The plan should specify what needs to be done, by whom, and in what time frame. It should also address the issue of what resources may be needed to effect the required change. A target date for re­audit should be set.

The findings of the audit should be shared with the entire clin­ical team, using effective channels of dissemination, which include face-to-face presentations, posters, and newsletters. The action plan should be discussed and agreed by the entire clinical team, so that everyone takes ownership of the project. This will facilitate any be­haviour change that is required in order to improve current practice.

The re-audit closes the audit loop. It entails repeating the audit cycle to see if current practice has changed as stipulated in the ac­tion plan and to demonstrate improvement.

The structure of an audit is simple (as described here) but the process of conducting it could be quite challenging. Implementing the action plan and bringing about change is usually the most diffi­cult part of the project. Some members of the clinical team may feel threatened by the project. Others may not see the need for change. Some may argue about the appropriateness, reliability, or applic­ability of the criteria or standards. There may be constraints of time, or the resources for implementing change may not all be available. The audit team should address these potential barriers proactively. Blame should be avoided and anonymity should be preserved when reporting audit findings. The focus should be on overall QI, not in­dividual failings (31).

Risk management

What is risk management?

Risk management is the totality of structures, processes, and behav­iours that are put in place to ensure that, as much as possible, clin­icians do the right thing every time; to mitigate damage from error; and to learn from safety incidents. The structures include com­mittee for overseeing the unit's risk management programme, data- gathering structures (e.g. electronic incident reporting system), and appointment of risk leads. The processes include formal mechan­isms for identifying risk, investigating incidents, and sharing les­sons learned. Behaviours reflect the safety culture of the unit and are best managed if the unit's approach to risk management is holistic, manifesting the attributes listed in Box 5.1. Risk is best managed not in isolation but within a framework that integrates all aspects of clin­ical governance. The RADICAL framework has been recommended for implementing and monitoring risk management (32).

Clinicians are, or should be, concerned primarily with ensuring that they do things right every time.

Patient safety incidents

A patient safety incident is an unintended or unexpected act or omission which caused or could have caused harm to a patient. A variety of terms have been used to describe such incidents or to reflect their severity. These include ‘adverse event', ‘serious incident',

Box 5.1 Attributes of a good risk management programme

• Comprehensive (includes all domains of the RADICAL framework).

• Evidence-supported (incorporates best practice, user experience, and available resources).

• Takes account of human factors.

• Intrinsic to clinical practice (not a stand-alone activity but applied as part of clinical care).

• Applies across professional and disciplinary boundaries.

• Visibly fair to all and promotes a just culture.

• Tailored to local requirements.

• Responsive to evolving challenges (such as structural or staff changes, and new threats to patient safety).

• Integrated with clinical audit and other quality initiatives.

‘serious, untoward event', and ‘critical incident. A ‘near miss' is an event which had the potential to cause harm but the harm did not materialize. A ‘never event' is a patient safety incident that is pre­sumed to be wholly preventable; for example, the retention of a swab or instrument in the patient's body or an operation conducted on the wrong person or wrong site. In reality, no incident is wholly prevent­able, as human error or violation of rules will occur at some point.

It is important to report patient safety incidents as part of the pro­cess of managing risk. For some events there are external reporting requirements—for example, all maternal deaths in the NHS have to be reported to the local Clinical Commission Group.

Identifying risk

All clinical areas should have formal processes for identifying any­thing that might interfere with the delivery of a safe, good-quality service. Identifying risk is, put simply, finding answers to the ques­tion ‘What could go wrong?' To answer this question, we could look back at patient safety incidents that have occurred and been re­ported in the department. The monitoring and analysis of reported patient safety incidents can reveal trends (upward or downward) and show areas of weakness or unsafe practice in the unit. Risk may also be identified from prospective risk assessments, patients' com­plaints, clinical audit, or concerns raised by staff. External sources of risk identification include national enquiries, national safety alerts, and findings of visitations by regulatory and monitoring agencies. Patients should be encouraged to report patient safety incidents, including near misses. Unfortunately, while this role of patients in reporting incidents is valued, robust mechanisms for promoting it have not been developed (36).

Risk assessment

This is a prospective and continual exercise to determine what risks lurk in the clinical area, the potential impact of these risks, and the effect of measures introduced to contain them. The potential impact is quantified by assigning a score derived from a risk matrix (Figure 5.4). The likelihood of the risk materializing is multiplied by the se­verity of harm that may be caused in the event that the risk material­izes. The assessment is incomplete without an action plan designed to contain the risk.

Risk registers

The risk register is a log of ranked risks that facilitates the moni­toring, review, and management of risks across the organization.

Figure 5.4 Risk scoring matrix.

Each clinical area should have a risk register. The major risks are escalated to an organization-wide risk register. The register includes controls that have been put in place to contain each risk.

Investigation of patient safety incidents

To learn from patient safety incidents we need to undertake a struc­tured analysis of what went wrong, what went well, what could have been done differently, and why events unfolded the way they did. This exercise is commonly referred to as ‘root cause analysis'. There are reservations about this term because although it has its merits (taking us to the factors underlying the incident), it may give the impression that there is one ‘root' cause and it fails to focus attention on systems improvement rather than finding of a cause. An alternative term used in the NHS, ‘high-level investigation', is also unsatisfactory—as it sounds inquisitorial. A preferable term is ‘system analysis'—an analysis aimed at unravelling the reasons why the system failed to deliver the quality care it was designed to deliver.

There are many approaches to the investigation of patient safety incidents. A widely used one is the London Protocol (or variants of it). This protocol offers a structured and systematic approach to the investigation of incidents (37). The key steps are as follows:

1. Identify incident and take decision to investigate.

2. Select members of the investigation team.

3. Gather data.

4. Outline the chronology of the incident.

5. Identify care and service delivery problems.

6. Identify contributory factors.

7. Devise an action plan.

The investigation team should have knowledge pertinent to the clinical issues being investigated and competence in conducting ac­cident investigation. This competence can be achieved by attending a training course. Investigators must be familiar with mechanisms underlying error, particularly the role of the human factor.

Data are gathered from health records, written accounts provided by staff, interviews, protocols, site visits, and any other means con­sidered useful by the investigating team.

Care delivery problems are proximal problems arising in the pro­cess of care, usually actions or omissions by members of staff, where care deviated beyond safe limits of practice and this deviation had at least a potential direct or indirect on the eventual adverse outcome. Service delivery problems are distal failures associated with organ­izational procedures and systems—for example, lack of supervision.

In analysing incidents, it is helpful to distinguish between ‘active’ and ‘latent’ failures. An active failure is the immediate cause of a pa­tient safety incident, for example, misidentification of a patient or sample; while a latent failure is a more remote but important cause, such as the absence of protocols for checking and confirming iden­tification. Care delivery problems are typically active failure while service delivery problems are latent failures.

Contributory factors are those thought to have some effect on the performance of individuals at the end of the error chain but are not directly causative of the incident. There is a wide range of in­ternal and external factors that could be contributory to a patient safety incident. The Yorkshire Contributory Factors Framework, which covers 16 domains, is an evidence-based framework that could be used to analyse contributory factors when investigating an incident (38).

As with audit, the action plan following an investigation should state in specific terms what needs to be done, by whom, and in what timescales; resource implications should also be addressed.

It is important to engage service users and staff in the course of the investigation. As discussed previously, any patient involved in a patient safety incident should be informed of what happened, and be given an apology and explanations. The patient, and where rele­vant their family, should be kept informed of progress with the in­vestigation, the findings, and the action plan. The staff involved in the incident should also be supported. They may suffer vicarious traumatization as a result of the incident, and this is more likely to happen if they have not had support from the hospital.

Unfortunately, evidence from multiple sources (15, 39-42) in­dicates that the quality and impact of incident investigations per­formed in the United Kingdom and elsewhere is generally of poor quality, and the ‘blame culture’ is still very much alive. There are many reasons for this, including the use of investigators who lack the appropriate training, insight, and competence required for incident investigation, the pressures of time, cultural barriers, and conflation of QI with litigation concerns. The first step in ad­dressing this problem is for units to incorporate into their clinical governance the continuous structured quality assessment of their investigation reports, using external assessors where possible. At na­tional level, a new United Kingdom safety investigation body, the Healthcare Safety Investigation Branch (HSIB), has been set up to act as ‘an enabler, exemplar and catalyst for learning-oriented safety investigation’ (43).