Surgical checklist use in Switzerland 2015 – where are we today?: a cross-sectional national survey study

Abstract

Background

Although surgical checklist use is not new in Switzerland, compliance and actual use fall short behind expectations taking scientific recommendation as standard. A national media campaign to raise awareness, inform experts, and change professional norms and standards on national level about checklist use was conducted. The aim of this study was to assess current checklist use in Switzerland following a national media campaign. We further analyse possible group differences between attending physicians, hospital staff, and participants of a quality improvement initiative.

Methods

A cross-sectional online-survey study was conducted by Swiss Patient Safety Foundation in Switzerland in 2015. The survey sample consisted of members of three Swiss professional associations of invasive health care (N = 1194). The survey assessed use of, knowledge of and attitudes towards the surgical checklist. A MANOVA to test for an overall effect and one-way ANOVAs for each dependent variable were conducted.

Results

For four out of six variables describing the ease of checklist use, hospital staff and participants of quality improvement initiative were significantly more positive about checklist use than attending physicians. A similar patter emerged for intentions, norms, attitude, acceptance, and perceived behavioural control. On all dimensions, hospital staff and quality improvement participants scored significantly higher than attending physicians. Significant differences especially between attending physicians and hospital staff and attending physicians and participants of the initiative emerged for different variables covering use of, knowledge of and attitudes towards the surgical checklist. However, effect sizes for all variables under study were small.

Conclusion

The results of the present study suggest that though WHO-surgical checklist use was further established in Switzerland it still needs to be promoted further, especially in outpatient care.

Background

In recent years, the WHO surgical safety checklist has emerged as a powerful intervention to increase safety in surgery. The checklist has become one of the most strongly recommended “single-intervention” safeguards worldwide [1]. If compliance is provided, the checklist has been shown to be an effective intervention to reduce morbidity and mortality in surgical procedures [2–8]. However, compliance with checklist use has repeatedly been shown to be less than perfect in different settings and different studies [7, 9]. Saturno et al. compared self-reported compliance rates with rates from direct observation. They found that self-reported compliance was about twice as high as compliance from direct observation [9]. This underlines that self-reported compliance rates often significantly overestimate actual checklist use. Approaching the lack of compliance by making the use of a surgical checklist mandatory, has not led to the desired comprehensive checklist use [7, 9–11]. Studies from countries in which checklist use has been made mandatory, exhibit no differences in actual compliance. Generally, implementing innovations via policymaking has been found to be rarely effective without accompanying resource intensive programs and effective change management [12]. One important aspect in changing health care professionals’ (HCPs) attitude and behaviour on the long run is to provide information to change individuals’ minds and beliefs. Innovations must be meaningful to an individual in order to enhance the readiness to change [4, 5, 12, 13]. In order to change professional norms on national level, information must be distributed and awareness must be raised via public relation work. Disseminating information about why and how the surgical checklist may improve patient safety is a vital aspect of a national media campaign aiming at individuals to adopt the checklist. Effects of system changes on national level are usually referred to as the “rising tide phenomenon”. Heightened public awareness of one specific problem within the health care system may lead to changes within the system as a whole which ultimately changes the baseline to start from [14]. This also applies to Switzerland. Although checklist use is not new in Switzerland, compliance and actual use fall short behind expectations taking scientific recommendation as standard. In a national online survey study in 2012, Mascherek et al. [15] found that about 80 % of the study participants in Switzerland used a surgical checklist on an everyday level. However, only 25 % reported using the WHO-surgical checklist. This underlines that checklist use remains unsatisfactory. The study also showed that 39 % of the nursing staff and 28 % percent of the doctoral staff were unsatisfied with the use of the checklist indicating a potential readiness to change. Dissatisfaction among the study participants could function as a strong motivational ground to change and improve checklist use. The 2012 study was conducted as national baseline before the start of a quality improvement program to further establish checklist use in Switzerland. Details on this national improvement initiative are reported elsewhere [16]. While the focus of the initiative was on 10 selected hospitals, the initiative was accompanied by measures of public relations. The national media campaign aimed at raising awareness, informing experts, and changing professional norms and standards on national level about checklist use. Mainly, the national media campaign promoted surgical checklist use as professional behaviour and vital tool to improve patient safety in surgical procedures. The surgical checklist was promoted as standard tool in any surgical procedure regardless of patient, specialty or urgency. 2.5 years after the national baseline study, we conducted a follow-up on national level to assess the status-quo of checklist use after the national improvement initiative. In the present paper we describe the results of the follow-up data-collection. We provide an overview of current checklist use in Switzerland after a national media campaign. Aim of the present study was to describe the actual status-quo of checklist use on national level in Switzerland. We further analyse possible group differences between attending physicians, hospital staff, and participants of the quality improvement initiative. Attending physicians mainly work in outpatient care and are only involved in hospital routines sporadically while conducting surgical procedures. However, even then they work largely independently from hospital routines as they use infrastructure and staff, but are often not involved in team aspects or work environment. As hospital staff individuals are designated who work exclusively in a hospital setting. They are tied to hospital culture, routines, and teams. Hence, they should be stronger affected by changes within the hospital routine. Finally, participants of the quality improvement initiative worked in hospitals which systematically introduced the surgical checklist as quality standard in every surgical procedure. Hence, those individuals were additionally exposed to explicit measures of the quality improvement initiative, including training, education and checklist implementation in their primary workplace throughout the duration of the initiative. We hypothesize that groups will differ as they were exposed to the national media campaign differently. While participants of the national improvement initiative were heavily exposed,, hospital staff in general was exposed via general public relations, however, tailored to their daily workplace. Least exposed, and probably least effected were attending physicians as they work self-employed in an outpatient setting, only being confronted with hospital routine when treating patients for a surgical procedure. As public relations mainly addressed staff working in inpatient care in acute hospital settings, we hypothesize that attending physicians should be least effected by the national media campaign.

Methods

Design

The cross-sectional online-survey study was conducted by Swiss Patient Safety Foundation in Switzerland in April and May 2015. The survey was programmed and administered via the EFS-Survey Software tool. The survey sample consisted of all members of three Swiss professional associations of invasive HCPs (doctors, nurses, nurses with special education in anaesthesia nursing or intensive care, and surgical technicians). Nursing staff was listed on a mailing list of all registered members of two professional associations. Physicians were members of the Foederatio Medicorum Chirurgicorum Helvetica. Subjects were invited to participate by individual emails in which aim and procedure of the study were explained. Participants obtained a personal URL to the online-questionnaire. Every URL could only be used once, hence, participation was only possible once per person. The survey was open for four weeks in total. Two reminders were sent by mail with an interval of 1 week and 2 weeks, respectively to non-responders. Completion of the questionnaire was regarded as informed consent. Ethical approval is not necessary for this type of study in Switzerland according to Article 1 and 2 of the Federal Act on Research involving Human Beings (Human Research Act, HRA) [17].

Survey

The survey was developed to assess use of, knowledge of and attitudes towards the surgical checklist. It was developed on the basis of extensive review of the literature and own pilot studies. Details on questionnaire development and translation of an earlier version of this questionnaire are reported elsewhere [15].The survey consisted of three conceptual and one socio-demographical part. The first part referred to the general knowledge and use of the WHO-surgical checklists (e.g., “Do you know the WHO-surgical checklist?”) and relative frequency of use for those using the checklist (rated on a 0-100 % scale subdivided into 6 categories). Satisfaction with checklist use was also assessed for those using it in part one and rated on a 5-point-Likert-scale ranging from “very unsatisfied” to “very satisfied”. In the second part, knowledge about the WHO-checklist was assessed (not reported in this analysis). Subjects rated their subjective knowledge about the checklist on a 5-point-Likert-scale from “very bad” to “very good”. In the third part, general attitudes towards the use of the checklist and the “ease of use” in clinical practice were assessed in a semantic differential. The semantic differential acts as “yardstick” and addresses the perceived atmosphere of checklist handling on different dimensions. The differential consisted of six pairs of adjectives with opposing meaning to describe checklist use. Paired adjectives were “easy-difficult”, “pleasant-unpleasant”, “familiar-unfamiliar”, “important-unimportant”, and “good for staff-bad for staff”, “good for patients-bad for patients”. Paired adjectives were rated on a continuum between 1 and 7, with 1 indicating agreement with the first and 7 with the second adjective. Numbers were not reverse coded for analyses. The continuum consisted of a 7-point-scale with “7” anchoring a positive attitude and “1” a negative attitude. Also, attitudes towards, acceptance of, norms, and intention to use the checklist were assessed. Norms (e.g. “Surgeons look down upon checklist use”) were measured with 4 items, acceptance (e.g. “The checklist is evidence-based.”) with 5 items, attitudes (e.g. “Checklist use enhances paying attention to patient safety”) with 7 items, and intentions with (e.g. “It is my plan to carefully mind the use of the checklist”) with 6 items. All items were rated on a 7-point-Likert-scale ranging from “do not agree at all” to “completely agree”. High scores indicate a positive evaluation of the checklist. Four items had to be reverse coded for data analyses. In the last part, socio-demographical variables were assessed. Also, two broad questions on patient safety in general (“Today, efforts to improve patient safety in surgery are much more prominent than a year before”) and the effectiveness of surgical checklist use (“Did you experience the prevention of an adverse event due to surgical checklist use?”) were addressed in this part.

Data analyses

In a first step, descriptive statistics were used to analyse checklist use and satisfaction with checklist use. In a second step, group differences between hospital staff, initiative-participants (“progress”), and attending physicians were analysed on different dimensions of ease of use, norms, attitudes, intentions, acceptance, perceived behaviour control and whether or not individuals had witnessed the checklist preventing an adverse event. Also, group differences concerning the question whether more is being done for patient safety today than a year ago were analysed. Although for some of the variables the assumption of homoscedasticity was violated, we conducted a MANOVA to test for an overall effect and one-way ANOVAs for each dependent variable as the sample was large enough and roughly balanced. ANOVA is a robust estimator against violations of homoscedasticity in large, balanced samples [18]. Scheffé tests were applied to correct for multiple testing. As a measure of effect size, ω² was reported for differences between groups. According to convention, 0.01 is considered a small, 0.06 a medium, and 0.14 a large effect [19]. All analyses were conducted using STATA v13.1 [20].

Results

Sample

Of the 4867 HCPs invited, 1194 completed the survey (24.5 % response rate). In total, 670 (56 %) worked as physicians and 524 (44 %) worked as nursing staff. Although the vast majority reported knowing the WHO-surgical checklist (1097 individuals, 91.9 %), this was not the case for 97 (8.1 %) individuals who were dropped from further analyses. Hence, analyses in the present study are based on 1097 individuals. 251 (22.9 %) participants worked in departments participating in the national quality improvement initiative, 667 (60.8 %) participants worked in a hospital not participating in the national improvement initiative and 179 (16.3 %) participants worked as attending physician. Detailed sample characteristics are presented in Table 1.

Table 1 Sample characteristics in per cent

Checklist use

In total, 754 (68.5 %) reported using the WHO-surgical checklist at their current workplace. Hence, 343 individuals reported not using the checklist at their current workplace (31.5 %). Of those 754 individuals using the checklist, 70 % reported using it for two years or longer. 85 % reported using the checklist at all or almost all (91-100 %) of the procedures. Only 69.4 % of the participants reported satisfaction with checklist use (see Table 2). Overall, 47.6 % report that they experienced the prevention of an adverse event due to surgical checklist use. Also, 85.2 % of the participants think that activities concerning patient safety in surgery have increased compared to a year ago.

Table 2 Description of checklist use in per cent

Group differences

Differences were analysed between participants of the national quality improvement initiative (“progress”), hospital staff (“hospital staff”) and attending physicians (“attending physicians”). Generally, effects between groups were small for all tested variables. Also, no differences between “progress” and “hospital staff” emerged. All significant differences emerged between “attending physicians” and either “hospital staff” or “progress”. For four out of six variables describing the ease of use, hospital staff and progress participants were significantly more positive about checklist use than attending physicians. Only for the dimensions “easy-difficult” and “good for patients-bad for patients” no significant difference emerged. Hence, attending physicians rate the ease of use lower than do hospital staff and progress-participants. A similar patter emerged for intentions, norms, attitude, acceptance, and perceived behavioural control. On all dimensions, hospital staff and progress-participants scored significantly higher than attending physicians. For “attitude” differences were significant only between progress-participants and attending physicians. No significant differences emerged between progress-participants and hospital staff on any dimension. Considering the question if patient safety in surgery had improved over the last year, agreement among progress participants was significantly higher than among hospital staff. No significant differences emerged between progress-participants and attending physicians or between attending physicians and hospital staff. Finally, progress-participants and hospital staff both reported significantly more often that they witnessed the checklist preventing an adverse event than did attending physicians. Again, no differences between hospital staff and progress-participants emerged. Although significant differences between groups especially between attending physicians and hospital staff and attending physicians and progress-participants emerged, effect sizes for all variables under study were small. Explained variance for each variable was between 1 % and 4 % (see Table 3 for detailed statistical parameters).

Table 3 Differences in mean scores of assessed constructs between groups

Discussion

Aim of the present study was to describe the status quo of the WHO-surgical checklist use in Switzerland after a national media campaign to promote checklist use in OR routine. Differences between three groups that were exposed differently to core aspects of the national media campaign were analysed. Use of other checklists than the WHO-surgical checklist was not assessed, because the national media campaign explicitly promoted the WHO-surgical checklist, as it is most commonly used today and most thoroughly studied. To date, the WHO-surgical checklist is widely used in Switzerland. Almost two thirds of the participants (63.2 %) report that they use the WHO-surgical checklist at their current workplace. Also, more than 90 % of the participants report that they are familiar with the WHO-surgical checklist. In 2012, only about 25 % of the participants reported using the WHO-surgical checklist [15]. Although this comparison is not based on repeated measures, it clearly indicates a more prominent use of the WHO-surgical checklist in Switzerland today. Results also show that individuals use the checklist frequently. 85 % of the participants report using the checklist in more than 90 % of the procedures. Checklist use also seems to be quite established today, as about 70 % of the participants report that they use the checklist at least since 2 years. The results reflect that for a majority of the participants use of the surgical checklist has become a professional standard. However, although checklist use seems to be widely established, still 37 % do not use the checklist. And of those who do, the checklist is still not used in 100 % of the procedures. These results clearly show that although surgical checklist use is widespread in Switzerland, room for improvement remains. As checklist use has shown to reduce morbidity and mortality only if compliance is high, systematic use still needs further promotion in Switzerland even today.

In a second step we analysed differences between initiative-participants, hospital staff and attending physicians with regard to attitudes, intentions to use the checklist and reported ease of use. Differences between groups were not confounded by profession (nurses and doctoral staff) as one might assume. Although overall effects were very small, significant differences especially between attending physicians and the two other groups emerged. Results reflected a generally more positive attitude, greater acceptance, and stronger intention to use the checklist in hospital staff and initiative-participants than in attending physicians. Also, hospital staff and initiative-participants reported significantly greater ease of use than attending physicians. Interestingly, no significant differences emerged for the reported value of checklist use for the patient. Although attending physicians evaluate the use of the checklist as less important and less pleasant, they nevertheless acknowledge the overall benefit for the patient. The results indicate that the checklist seems to be accepted as a useful tool for patient safety in surgery across all HCPs. However, actual using the checklist and accompanying behavioural aspects were less established and less positively evaluated in attending physicians than in initiative-participants or hospital staff. The same was true for general attitudes and the intention to use the checklist. Here, again, initiative-participants and hospital staff reported significantly more positive attitudes towards the checklist and a stronger intention to use the checklist than attending physicians. Taken together, one may conclude that although the overall evaluation of the benefit of the checklist for patients is similar across groups, initiative-participants and hospital staff report more positive attitudes and greater ease of use. Maybe due to the greater exposure to situations with checklist use, initiative-participants as well as hospital staff also report significantly more often than attending physicians that they witnessed the prevention of an adverse event due to checklist use. The prevention of an adverse event might be one factor that additionally influenced the attitude of hospital-staff and initiative-participants in comparison to attending physicians. Witnessing the prevention of an adverse event might be most influential for future checklist use. Interestingly, the only difference that emerged for the question concerning whether or not patient safety was more focussed on today than a year ago emerged between initiative-participants and hospital staff. Initiative-participants reported significantly more often that patient safety received more attention than year ago. This might be due to the fact that participation in the initiative was resource intensive for individuals, hence, patient safety in surgery as a topic was very present throughout initiative duration.

This study has several limitations. First, generalizability of the findings is limited. Although for doctoral staff, all members of the professional associations of invasive HCPs and for nursing staff, two large professional associations were addressed; the response rate of about 25 % limits the significance of the results. Additionally, for nursing staff, members of a third relevant professional association could not be addressed for data protection reasons. Quantitatively, the response rate is in accordance with other studies in health care [21], however, we cannot rule out sampling bias or response bias as no information about non-responders were available. Second, results are not based on repeated measurement. Hence, although we discuss development of checklist use in Switzerland, no real inferences can be made. Although unlikely, we cannot rule out the fact, that in a worst case scenario differences are due to sample composition alone and not to development in checklist use. Third, no causal inferences about the impact of the national media campaign on checklist use can be made. Although differences found in the data point into the direction hypothesized, it is not warranted to causally assign them to the national media campaign. Other influences as well as systematic differences between groups that were not assessed could be alternative explanations of the results. Fourth, overall effects are very small. Hence, although differences between groups exist statistically, practical relevance is limited.

Conclusion

The results of the present study suggest that WHO-surgical checklist use in Switzerland today is more established than in 2012. Differences in attitudes towards checklist use and evaluation of ease of use emerged between hospital staff, initiative-participants and attending physicians. These differences underline the assumption that promoting surgical checklist use in an outpatient setting is even more complicated than within the very structured setting of a hospital. Future efforts need to explicitly address health care professionals working in outpatient care in order to further establish surgical checklist use in Switzerland. Taken together, the results of the present study show that although WHO-surgical checklist use was further established in Switzerland it still needs to be promoted further, especially in outpatient care.