Article Text
PDF
Abstract
Background Safety rules continue growing rapidly, as if constraining human behaviour was the unique avenue for reaching ultimate safety. Safety rules are essential for a safe system, but their multiplication can have counterproductive effects.
Objective To monitor, in an anaesthesia ward, compliance with a process-oriented safety rule, and understand barriers and facilitators which help and hinder physicians from following guidelines.
Methods The rule stipulated that the day before surgery anaesthetists had to record in the patient's file the drugs to be used for the anaesthesia (induction, maintenance, airway control). Compliance was assessed before introduction of the rule, immediately after, at 6 months and at 12 months. All medical staff were blinded to the protocol.
Results 717 patient records were included. The results showed an initial compliance with policy, reaching 86% for some items (never 100%). Reduction began within 6 months and returned almost to initial levels within a year. One individual showed poor compliance throughout the study but even initially compliant doctors experienced a reduction. Compliance was higher for complex surgery but lower for unscheduled surgery and when job pressure was greater.
Conclusions Compliance eroded over time. A major trigger of erosion seemed to be lack of continued compliance by a senior member of staff. Rules and procedures constitute fragile safety barriers, and it may be better to forego introducing a new safety rule if it is not considered as a priority by staff and is therefore vulnerable to sacrifice in case of conflict with competitive demands.
- Compliance
- rules
- migrations
- violations
- error
- teamwork
- human error
- safety culture
- guideline
Statistics from Altmetric.com
Introduction
Safety rules, protocols and recommendations are defined ways of behaving intended to either achieve or improve on a required level of safety.1 Safety rules and recommendations continue growing rapidly, as if constraining human behaviour were the unique avenue for reaching ultimate safety. Safety rules and standards, especially when evidence-based, are of course an essential foundation of a safe system. However, the multiplication of safety rules and procedures, and the inclusion of more and more process-oriented rules, can potentially have counterproductive effects. For many safety rules, evidence remains weaker than for the new clinical interventions. Interventions to improve the quality of handover or to reduce wrong site surgery seem reasonable and sensible. They can show changes in clinical processes, but for a variety of reasons, it is hard to link such changes directly to improvements in clinical outcomes. Many of these procedures are certainly implicated in safety incidents but, by themselves, are not the sole cause. Interventions to improve such clinical processes may be valuable in reducing the overall level of risk in the system, although vulnerabilities in such processes are seldom by themselves sufficient to cause patient harm. Nevertheless, such rules tend to be acceptable, provided the intervention is low cost, easy to implement and with low risk of harm; then, the strength of evidence need not be high to justify implementation. The decision usually refers to this as the ‘proportionality of burden of proof.’2
The main problem with rules is not so much their potential intrinsic value but compliance with them. The compliance with even proven safety recommendations remains low.3 The problem of compliance may be much greater with rules without a clear evidence base and have no immediate negative feedback on the patient when not followed.
Non-compliance to safety rules is, however, less easy to define, as the expected level of compliance depends on the type of instruction, the nature of the work, and the social and organisational context. In healthcare, for instance, evidence-based medicine is the source of guidance for practice but not of mandatory guidelines.4 In some cases, therefore, strict observance is understood to be necessary, whereas, in others, some flexibility is tolerated or even expected. However, serious violations are likely to be severely penalised in all circumstances. Although there are many degrees of non-compliance, it is usually said in the medical literature on safety culture (David Marx, Robert Helmreich) that human error or negligent conduct is forgivable, but intentional rule violations are reprimanded.5 6 Note that in many cases, non-compliance is passive (not actively following the rule) rather than explicitly doing something unauthorised, and thus may appear to be a much lower degree of violation. Whatever the type of non-compliance (not doing/doing differently), the difference between intentional and non-intentional rule violation is a matter of sensitive interpretation, from the worker, and even more from the investigation boards and the justice. The study of non-compliance is thus particularly challenging, as it concerns a sensitive issue not open to usual lines of enquiry.
We know some barriers and facilitators that help and hinder physicians from following guidelines. For example, Grol and Wensing made a survey of physicians in The Netherlands on perceived barriers to implementing guidelines on diabetes care. Among all contributively factors, they identified a few individuals' cognitive factors (not fully understood), a chronic poor design of guidelines (too rigid, cost too much time) and some organisational factors (no support by management, competitive tasks, etc).7 Similar factors have long been identified in industry. ‘Intentional non-compliance’ by aircrews accounts for 45% of all errors and violations, but only 6% of these affect flights.8 Half the checklists on airplanes are incorrectly completed because of interruptions and poor checklist design.6 Violations of procedures may even increase as industries become safer.9 The very narrowness of the rules governing high-safety systems and the increasingly lengthy and time-consuming checklists developed in response to successive adverse events make violations more likely.6–9
Several models of violations have been reviewed recently.10 Rasmussen has argued that front-line workers do not follow strict procedures but follow the most useful and productive path.11 The Rasmussen/Amalberti framework derives originally from Rasmussen's framework model of system migration and attempts to integrate the various perspectives on violations and to resolve some of the apparent contradictions between them.12 Workers operate within an envelope of possible actions and are influenced by organisational and social forces. Violations become more frequent and serious with time so that the whole system ‘migrates’ towards the boundaries of safety. The Rasmussen/Amalberti model posits that erosion of compliance and migration start soon after new safety barriers are introduced, that is, when the safe space of action defined by these barriers curbs the performance of the system. The model predicts that (1) the system will start migrating when, paradoxically, those who imposed the safety rule request better performance, (2) the deviations will not be perceived as an immediate danger because of the wide margins for error in a well-constructed system, and (3) individual workers will take advantage of the system by cutting corners and setting new boundaries. If this silent migration is not detected and controlled, it may contaminate all types of compliance including compliance to mandatory rules and thus cause harm.
The Rasmussen/Amalberti model has been tested in several environments,13–15 but the mechanisms of erosion of compliance and migration of a system over time are incompletely understood, especially the factors that lead to non-compliance by ‘not doing any action’ in the case of process-oriented rules. We have therefore explored the gradual erosion of compliance with a new safety rule over the 12 months after its introduction on an anaesthesia ward. Anaesthesiology is a specialty involving considerable experience in the use of standard protocols, the implementation of new guidelines and the traceability of clinical activity.
Methods
Setting and patients
This prospective cohort study (January 2004 to January 2005) monitored compliance with a new safety rule introduced by the Chief Anaesthetist on the anaesthesia ward of the Percy military teaching hospital (about 4000 anaesthesias/year). Since 1994, each patient in France must by law meet an anaesthetist at least 2 days before surgery, and also be seen by an anaesthetist the day before surgery.16 According to the French Society of Anaesthesiology (SFAR) guidelines, the anaesthesia procedure should be carefully planned before surgery.17 This legal background explains that it is already implicit for anaesthetists to write plans in the patient file.
The rule introduced by the Chief Anaesthetist goes a step further from the generic recommendation of the SFAR, asking for a longer list of items to be detailed the day before anaesthesia: drugs used for induction of anaesthesia, anaesthesia maintenance strategy and method of upper-airway control (intubation, facial mask, laryngeal mask). The rule belongs to a growing category of rules that go well beyond basic safety requirements and that aim to develop excellence in practice. The reason for this was that the same anaesthetist may not perform the presurgery visit and be present during surgery. The assumption was that implementation of the rule might reduce the incidence of adverse events.
All staff (six senior and three junior anaesthetists, 19 nurses) were blinded to the study protocol during monitoring except the two investigators (YA, a senior anaesthetist, and GSM, a junior anaesthetist). One senior anaesthetist was absent during the study. All staff signed the new protocol. No feedback was given, and no reminders were issued during monitoring.
All patients undergoing general anaesthesia were prospectively included during one of four study periods, before introduction (P1, 3 weeks) and after introduction of the new rule: immediately after (P2, 2 weeks), 6 months after (P3, 3 weeks) or 12 months after (P4, 3 weeks).
Data collection
Data from patients' records and the ward database were collated and reviewed by either YA or GSM. Recorded variables were: drugs prescribed for anaesthesia induction (hypnotic, opioid, muscle relaxant), anaesthesia maintenance and airway control (from the anaesthetist's presurgery record), patient age, gender, American Society of Anesthesiologists (ASA) score, scheduled or unscheduled surgery, duration of surgery, incidents during anaesthesia, names of anaesthetists (presurgery visit, during surgery), day of the week (surgery) and position on the surgical operating list.
Statistical analysis
Each documented item (ie, each drug, maintenance, airway control) was compared independently between the study periods (P1 vs P2, P2 vs P3, P3 vs P4). Compliance was analysed in a multivariate logistic regression model. The dependent variable was the presence/absence of the item in the patient's file, and the explanatory variables were individual and organisational factors thought to trigger erosion or migration. The factors analysed for their influence on compliance were: individual compliance of four doctors blinded to the study (three senior and one old junior), teamwork (the same vs a different anaesthetist before and during surgery); complexity of surgery (first operation of the day—usually the most complex—vs surgery later in the morning); social and job pressure (surgery scheduled for a Monday vs other days of the week; Monday surgery supposes a previsit on Sunday, hence testing the resilience of compliance in degraded conditions); emergencies (unscheduled vs scheduled surgery); and difficulty of the case (duration of surgery). All factors were studied by univariate and multivariate analysis for each documented item.
Interviews and debriefing
At the close of the study, the investigators revealed the study to doctors and showed their results over time. Each doctor was asked for permission to use the data for publication, and they were interviewed about their motives for progressive non-compliance. We conducted non-directive interview of the four senior anaesthetists using their records as supporting material, in order to discover which factors had in their opinion triggered erosion of compliance.
A debriefing was then conducted with all staff members during which the study's findings were presented.
Results
Sample characteristics
A total of 907 patient files were examined. Of these 907 patients, 717 had undergone general anaesthesia. There were no significant differences in the clinical characteristics of the patients according to inclusion period. None of the adverse events recorded was related to compliance with the new safety rule.
Compliance
Figure 1 shows how often the requested items were recorded in the patient's file during the four study periods. Before the new rule was introduced (P1), mention of the anaesthesia-inducing drugs to be given was made in 70–80% of the files. Anaesthesia maintenance and airway control were, however, mentioned in only 40–45% of files. The mean overall compliance was 62%. In the 2 weeks after the new rule was introduced (P2), the mean compliance increased to 78%, with a maximum compliance of 86% for mention of the muscle relaxant. The mean compliance remained relatively stable at 6 months—the slight downward trend observed was not significant (P3 vs P2)—but fell sharply between 6 and 12 months (P4 vs P3, p<0.05). By 12 months, the mean compliance for mention of induction drugs had very nearly dropped to its initial level.
Compliance with rule over 1 year (P1, before introduction; P2, immediately after introduction; P3, at 6 months; P4, at 12 months).
Factors affecting compliance
When comparing compliance before (P1) and immediately after (P2) introduction of the rule, we found that compliance was significantly greater for the first operation of the day but reduced for unscheduled surgery, as illustrated in table 1 (P1 vs P2) for the hypnotic to be given. A multivariate analysis of other items (other drugs prescribed, airway control and maintenance) gave similar results.
Factors affecting compliance with the instruction to record the hypnotic to be used for induction in the patient's file
These same factors (first and emergency operations) were also significant in a comparison with data at 12 months, indicating that they remained influential over time (table 1, P2 vs P4). Interestingly, an additional factor, namely the day of the week (Monday), also proved to be significant at 12 months.
A comparison of compliance at P2 and P4 revealed substantial differences in individual compliance (table 2). Anaesthetists 1, 2 and 4 substantially increased their compliance when the rule was introduced and more or less maintained it at 6 months (not shown). However, by 12 months, the compliance of Anaesthetists 1 and 2 had fallen. Only Anaesthetist 4 continued to do exceptionally well. Anaesthetist No 3 was poorly compliant with the rule throughout the study, showing little interest in group activities and a probable reluctance to waste time providing information considered to be ‘nice to know’ but not very high priority (as he readily admitted himself during the interview).
Individual compliance with the instruction to record the hypnotic
Interviews and debriefing
Anaesthesiologist 3 acknowledged that he had not understood the added value of the rule and so had continued following past routines. The three other anaesthesiologists recognised the potential value of the rule, but found many reasons to justify their progressive decrease in compliance over time, sacrificing or shortening previsits to satisfy more urgent competitive duties (enrolment in unexpected medical task, change in the planning, longer time spent in the operating theatre, etc), or not applying the rule in several contexts where it appeared to be irrelevant (Sunday, emergencies, etc). Examples of reasons for non-compliance given by staff after disclosure of the results of the study are given in box 1.
Box 1 Typical staff comments after disclosure of study findings
Do you remember cases where you did not comply with the rule? Please say why:
Doctor X: Yes, it occurred several times. The nurses wanted to know what was the anaesthesia protocol for the next patient. But I need to see the file to tell them, otherwise I don't remember. In most cases, the file comes with the patient, then the nurse makes a standard preparation, and we keep it, since it is here. The problem is that we only get the file with the patient. We used to save time by preparing for the next patient …
Doctor Y: I cannot just follow the instructions of my colleague who made the presurgery visit. I am a doctor, and I must make my own judgements, and be in control of what I do.
Nurse X: I was an adviser to the doctor before, now they want me to become a syringe shooter. I tell you that I will not accept this rule.
Nurse Y: There is no problem complying with the rule on the morning of surgery. But in the afternoon, if there is a patient you want to wake up quickly after a short operation, you will not for example give Pentothal Sufenta—Tracrium but Diprivan—Sufenta.
Discussion
The introduction of the new rule was initially accompanied by a marked increase in compliance. There may be several reasons for this. Ours is a typical anaesthesia unit, but it focuses strongly on training staff in best practice and has a longstanding history of incident reporting and of weekly seminars on safety and technical issues. In addition, the new rule derived from SFAR guidelines meets the need for greater traceability of actions in patients' records. According to our hospital's accreditation report, our anaesthesia unit is ranked among the best in the country with regard to traceability.18
Compliance was still high at 6 months after introduction of the new rule but dropped to almost its initial level at 12 months. The rule was normally introduced, known by staff, sometimes evoked in informal meetings and coffee breaks, but never re-enforced in official briefings. The chief anaesthesiologist never intervenes specifically during the year into staffs or person-to-person briefings to show and comment on adverse events related to non-compliance with the rule (none were noted), or to comment on the gradual poor level of compliance.
Its lifespan as an ‘effective intervention’ was thus less than 1 year.
We identified five factors that might have accounted for erosion of compliance:
The nature of the new rule. The rule belongs to a growing category of process-oriented rules justified by preventive risk analysis, expert judgement and experiential reports, but not by strong epidemiological evidence. They are usually accepted because the intervention is low cost, easy to implement and with a low risk of harm; in these circumstances the strength of evidence need not be high to justify implementation. Medical staff tend to consider many of these rules to be an extra burden on an already overloaded safety agenda. If non-compliance does not cause any immediate adverse events, these rules may easily be pushed aside by other priorities. The absence of feedback on the efficacy and shortcomings of the rule probably also signals to staff that the rule is not a priority.
Pressure of work and competing priorities. After 6 months, compliance was significantly lower when the patient was admitted to the unit on a Sunday. On Sundays, a single anaesthetist has to visit all the patients on the Monday's surgical lists (more than on other days) and deal with all emergencies. The pressure of work (individual factor) and lower team participation (organisational factor) reduce compliance, and poor compliance on Sundays could generate poorer compliance at other times. A group discussion after an anaesthetist had refused to return on a Sunday to see a non-critical patient who arrived late convinced staff that the new rule posed problems and was not absolutely necessary. This event probably affected the compliance of all the doctors.
Exceptional cases. In general, compliance was poorer for unscheduled than for scheduled surgery throughout the study. Although the rule was not meant for true emergencies, it could actually have been applied in all exceptional situations. Since non-compliance in these situations carries no clinical or social penalties, the temptation is great to cut corners and violate rules. Similar results have been obtained in the aviation and printing industries.6 13 14
Case difficulty. Compliance was higher when the patient was first on the surgical list. The first operation in the morning is often the most complex or risky and attracts the anaesthetist's full attention. Patients at the bottom of the list with less serious conditions may receive only a rapid, perhaps cursory, visit from the anaesthetist. Throughout the study, compliance was lower for routine, simple cases than for difficult cases. It became worse with time, indicating that transgression in low-risk cases may spread and diminish overall compliance.
Individual motivation. Substantial individual differences in compliance were noted, and this might have affected overall compliance. The staff had a great respect for the Chief of the department and an established culture of adherence to safety rules and procedures, two factors that stimulated initial compliance. However, the failure of an initially compliant senior anaesthetist to continue complying may have prompted others to be less compliant also. That is typical of the influence of leaders as described in crew-resource management courses in Aviation.19 20 A second senior anaesthetist showed poor compliance throughout the study. However, the impact may not have been as great, as his attitude was probably seen by others as consistent with his overall outlook. It is known that staff shortages encourage craftsman attitudes, excessive tolerance and non-compliance, leading to diminished overall control.7 21 22 Individuals confronted with their unconventional behaviour who are encouraged to take part in teamwork can be brought into line. Unfortunately, the usual response is covert criticism.
Of the five factors, one thus concerns leadership. Three others are due to lack of perceived relevance. The new rule either did not apply (as in emergencies) or was maybe of insufficient clinical benefit (one doctor cited this hypothesis).
Our study has thus confirmed the predictions of the Rasmussen/Amalberti model. The model predicts that migration will be greater and faster when the rules impact on the performance of the system, especially if these rules have limited visible immediate benefits on safety. This is often the case for ‘nice-to-follow rules’ such as those in our study where compliance is just a question of paper-and-pencil and traceability. It may be less true for ‘need-to-follow’ rules where compliance tends to be a more technical matter. The model also predicts that managers will end up requesting local non-compliance to make sure the job is done well and in good time. Our senior anaesthetist may have taken on this role when he decided to implement the rule no longer (not coming for previsits scheduled on Sunday). Finally, the model predicts that, after initial non-compliance, all individuals are quickly contaminated by migration of practice. At 12 months, the compliance of all our anaesthetists (except one) had fallen to virtually its initial level.
In conclusion, introducing a mixed bag of ‘nice-to-follow’ and ‘need-to-follow’ safety rules into a readable and stable safe system is to be avoided. Compliance with the ‘nice-to-follow’ rules will be eroded first. However, the mere presence of these rules and the multitude of targets make the system clumsy. In addition, the erosion of compliance with these rules, as well as migration, can affect compliance with all rules including ‘need-to-follow’ rules.23–25 When introducing a new policy or procedure, therefore, it is important to start by identifying potential barriers and having realistic expectations about compliance. The right attitude might even be to decide against the introduction of policies with a limited, even if proven, capacity of safety improvement, especially into systems under pressure. Group discussions are helpful in identifying potential difficulties and establishing when compliance can be expected and when some deviation can be tolerated. However, compliance will gradually erode if there is no peer control, and some individuals will drift towards truly dangerous behaviour. Peer control then becomes more difficult though not necessarily ineffective. Violations may continue for months or even years. Needless to say, dangerous violations causing adverse events mean that effective action has been left too late. A severe adverse event can lead to tightening of the rules, but there is also the risk that the individual will be disciplined or even removed, while leaving the culture of lax practice and violations intact.26
The main limitation of our study is the small number of participants and cases. However, our findings probably do reflect the problems of the introduction of ‘nice-to follow’ policies. Larger, multicentric studies are needed to confirm these findings but also to improve our understanding of the erosion of compliance with ‘need-to-follow’ rules. This would require a complex longitudinal behavioural study with long-term access to data, blinding with an ethical attitude towards participants and the full trust of staff before and after the study so that near-misses not requiring official reporting (unlike serious adverse events) are disclosed.
Key messages
When introducing safety rules, the worst scenario is probably to introduce a mixed bag of ‘need-to-follow’ and ‘nice-to-follow’ rules into a readable and stable safe system. Compliance with the ‘nice-to-follow’ rules will erode first, but these rules can feed migrations and contaminate compliance with all types of rules, including the more important ‘need-to-follow’ rules.
Paradoxically, it may be beneficial to decide against introducing safety policies that have a proven but limited capacity for safety improvement, especially into systems under pressure.
Perfect (100%) compliance with a rule is not realistic. The criteria for non-compliance should be a matter for a group discussion and not be left to the individual. Violations cannot be eliminated but can be managed. Our vision of safety is that of an ideal world of reassuringly clear rules and procedures. However, the barriers formed by these rules may be fragile. We seldom test them in out-of-the-ordinary situations—for example during weekends rather than work days or with less highly qualified staff than usual. It is necessary to understand first the pattern of violations and system migration, and then gradually to try to influence staff behaviour within the broader system.
The largest gaps in compliance are those that remain undetected because indicators are lacking. Organisational migrations and one's own migrations are the hardest to see.
Practice should be monitored and deviations recorded before the first incidents or accidents occur. System migration can be controlled and managed by introducing observation periods and instigating open discussion on safety roles within the group.
Acknowledgments
We thank T Ojasoo for her helpful comments, suggestions and English editing.
References
Footnotes
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- Quality lines