Analysis of costs and benefits of a re-accreditation of a Swiss acute care hospital

Abstract

Background

Accreditation of hospitals and other institutions is a widely used instrument for the quality assurance in health care. However, relevant literature regarding the economic evaluation of hospital accreditation is still missing. To date no formal Cost-Benefit Analysis (CBA) or Cost-Utility Analysis (CUA) has been carried out.

Methods

This study uses an existing framework specifically developed for the economic evaluation of hospital accreditation. Based on this framework, we identify and quantify the costs and selected benefits of the re-accreditation of a Swiss acute care hospital. Costs are identified and quantified by conducting key informant and expert interviews. Benefits are identified by hospital experts and rated using a newly developed validation tool.

Results

Costs of the re-accreditation amount to about CHF 870'000 (internal and external costs). Benefits in quality management (QM) and in the critical incident reporting system (CIRS) are quantified and rated by the hospital experts in the following order: (1) development or promotion of a quality or safety culture, (2) implementation of a hospital-wide complaint management, (3) fulfillment of the hospital vision, (4) improved image upon stakeholders (patients, suppliers etc.), (5) improved image in policy, (6) quality dashboard, (7) preparation of centralized quality documents, and (8) avoidance of liability cases.

Conclusion

This study provides detailed information about costs and selected benefits associated with the re-accreditation of a Swiss acute care hospital. As opposed to the costs, benefits could not be monetized but were quantified using an expert rating to illustrate the impacts of the re-accreditation. Overall, our study confirms the difficulties in the economic evaluation of hospital accreditation, but it makes a step towards a formal CUA.

Background

The number of accreditation programs and accredited hospitals has increased considerably over the last decades. In 2000, the World Health Organization identified as many as 36 nationwide accreditation programs and the number of programs has doubled every five years since 1990 [1]. In the United States, accreditation is a requirement for reimbursement by public health programs and over 85 % of the hospitals are accredited [2]. In Switzerland, the focus of this study, there are different accreditation programs in place, e.g., on the accreditation of entire hospitals, subdivisions or centers [16].

Persistent harm during health service delivery is an issue [3, 4]. Moreover, accountability and safety in health care has increasingly become an international concern [4]. In this regard, accreditation programs aim to improve the quality of care [5]. On the downside, achieving accreditation generates significant expenditures [6]. And whether accreditation contributes to a quality improvement is still controversial. Some argue that it does, in the sense of providing a useful tool to stimulate improvement in health service organizations [7] and in the processes of care and clinical outcomes [8]. In contrast, Grepperud [2] argues that there is no convincing evidence for quality improvement.

Despite its wide international implementation and emerging importance, relevant literature seems to be missing especially regarding the economic evaluation of hospital accreditation. For this reason, more research in the area of health service accreditation seems to be well-justified [1, 4, 7, 9, 10]. To gain a better understanding of the impact of hospital accreditation there is a strong need to economically evaluate such efforts. This study attempts to deliver a basis for further in-depth analysis in this context.

Related literature

So far, several studies have tried to assess the impact of hospital accreditation. However, we are not aware of any formal economic evaluation in the literature, which is confirmed by Mumford et al. [6] who conducted a systematic review on the use of economic evaluation techniques in connection with health services accreditation. Nevertheless, there exist different attempts to analyze the impact of hospital accreditation that are mainly based on meta-analyses, single cost or benefit studies.

Greenfield and Braithwaite [10] systematically reviewed 66 articles that examine health sector accreditation. Their review included the impact of accreditation on specific topics. For some topics they found consistent, for others inconsistent evidence and for some they were not able to draw any conclusion. A narrative review of 122 articles (categorized into key thematic topics) by Hinchcliff et al. [7] confirmed the multifaceted picture. However, the study also identified knowledge gaps concerning accreditation research (e.g., on the relationship between accreditation and suitable quality measures and indicators, or on the financial impact of accreditation of health services). Braithwaite et al. [11] analyzed the association between accreditation results and clinical or organizational performance measures. The findings were not entirely conclusive but indicate that a better accreditation performance was associated with an improvement in quality measures.

As an example of a cost study, Zarkin et al. [12] estimated the costs of preparing and undergoing accreditation of a methadone treatment site using survey data. They estimated total costs of accreditation as $48'005 which equalled to about 5 % of the clinic’s annual operating costs. Sack et al. [13] studied the association between hospital accreditation and patient satisfaction with hospital care. The authors could not find a link between hospital accreditation status and improved patient satisfaction (benefit). Grepperud [2] focused on the decision to accredit a hospital and its social effectivity, which may be seen as a different approach to evaluate accreditation performance.

Most related to our study is Mumford et al. [4] who intend to apply a formal Cost-Benefit Analysis (CBA) on acute care accreditation. This is a current, ongoing study based on a newly developed framework called SIQNS (Scope and objectives, Identification, Quantification, Net present value, Sensitivity analysis). This framework will be presented in greater detail in the methods section below. Finally, Mumford et al. [14] reviewed the suitability of hand hygiene as a possible quality indicator of accreditation outcomes. Data on hand hygiene compliance rates and hospital accreditation scores for 96 hospitals were analyzed, but at the end the suitability of the indicator was denied.

Quality assurance in the Swiss healthcare system

In the Swiss health care system, each of the cantons is responsible for the planning and provision of care. Nevertheless, federal law is regulating some parts of the health system, which also affects the regulation of quality assurance. All health care providers in Switzerland are obliged by law to ensure the quality of care. However, each health care provider is self-responsible for its quality assurance, and the accreditation and implementation of a Quality Management (QM) system is voluntary [15].

Accreditation bodies

The Joint Commission (United States) is one of the most prominent accreditation providers in the world. However, European accreditation programs have emerged over the years and accounted for about half of all programs worldwide in 2003 [1]. On top of all the national and international accreditation bodies, the International Society for Quality in Health Care (ISQua) serves as the peak body [4]. The ISQua defines accreditation as “a public recognition of the achievement of standards by an organisation demonstrated through independent assessment in relation to set standards”.

The term accreditation in Switzerland is often used to state an acceptance of organizations by associations or the state [16]. In the context of peer-review assessments, the term “certification” is more common. Nonetheless, we will use “accreditation” when relating to peer-review assessments consistent with the international literature.

The focus of our study lies on the accreditation body sanaCERT suisse (sanaCERT). SanaCERT is a Swiss foundation (non-governmental organization) founded in 2001. The sanaCERT accreditation program is similar to the programs of the Joint Commission, Accreditation Canada and the Australian Council on Healthcare Standards (ACHS). SanaCERT aims to review the achievement of its quality standards during on-site audits by using the concept of peer reviews [17]. The objective of the audit is to examine the conformity with the standards. Each health care institution chooses a set of standards they intend to focus on, except for QM, which is mandatory and cannot be dropped. To meet the requirements of the chosen standards the institution has to provide documented information as well as oral statements. In the case of a sufficient result, the institution receives an accreditation status from sanaCERT. Re-accreditation takes place every three years under the condition of yearly routine surveillance visits.

Cantonal Hospital of Graubünden

The cantonal hospital of Graubünden (KSGR), a 355-bed (only acute-care) teaching hospital operates as a centralized hospital in the south-eastern region of Switzerland. In total, the hospital has almost 1900 employees and the revenue amounted to just over 300 Mio. Swiss Francs in 2013 [18]. The hospital was for the first time accredited by sanaCERT in 2011. After three years, the re-accreditation took place in November 2014. The KSGR has decided on eight standards: QM (main standard), survey of patients’ opinion, surgery, Critical Incident Reporting System (CIRS), employed people and hospital, emergency, nutrition and safe medication, which were examined by sanaCERT for the re-accreditation.

Scope and objectives

Based on the SIQNS framework by Mumford et al. [4], the first aim of this study is to identify the costs and benefits associated with the re-accreditation of the KSGR. Quantification and monetization of benefits in general is a difficult task. For this reason, the study’s second aim is to elaborate whether it is possible to quantify selected benefits. To this end, we develop a new validation tool that allows hospital experts to rate benefits. Finally, as there is few literature available on the economic evaluation of hospital accreditation, every contribution concerning the topic may be useful. Hence, our third aim is to discuss directions for further research and more detailed analyses based on our results.

The framework by Mumford et al. [4] is used as a basis to answer the following research question: What are the costs and benefits of the present re-accreditation of an acute care hospital and is it possible to compare them in a formal cost-benefit analysis in this setting? It is not intended to answer the question as to whether the decision to re-accredit was an efficient one. This is a different topic, as we can see from Grepperud [2].

Methods

Mumford et al. [4] introduced the SIQNS framework for the economic evaluation of accreditation. The model and its methodological steps are summarized in Fig. 1. The SIQNS framework was created for assessment by different institutions [4]. This section is devoted to describing the methodological procedure according to the SIQNS framework, focusing on the first three parts (scope and objectives, identification, and quantification).

Fig. 1

SIQNS Framework by Mumford et al. [4]

Step 1: Scope and objectives

Scope and objectives of our study were stated in section 2. We also provided a review of the related literature and background information on hospital accreditation as well as the relevance and aims of our study, as suggested by Mumford et al. [4].

Step 2: Identification of costs and benefits

The second step of the SIQNS framework comprises the identification and recognition of costs and benefits. Benefits are understood as a general concept here, representing any improvement/change of an indicator in the hospital environment, which for example can be the improvement of processes, documentation or safety aspects. We approached the identification of costs and benefits by conducting key informant interviews. The following standards were selected for deeper review based on expert opinion: QM, surgery, CIRS, and nutrition. QM and CIRS cover hospital wide projects, which in turn represent hospital wide costs and benefits of the re-accreditation. The other two standards are more narrow and provide department specific costs and benefits of the re-accreditation. Once the respective standards were selected, each project leader from the four identified standards was contacted and interviewed using a semi-structured interview guideline (Additional file 1). The aim of the interviews was to identify standard specific costs and benefits.

To identify the costs, interviewees were asked to report the expenses and other costs that were documented. If costs were not documented, experts were asked to estimate them. Additionally, the hospital quality manager estimated the costs of the standards not selected for deeper review: survey of patients’ opinion, employed people and hospital, emergency, and safe medication according to Mumford et al. [4] to cover the costs of the entire project. If it was not possible for the quality manager to estimate the costs, the project leaders from the respective standards were contacted to collect the missing information.

To identify the benefits, interviewees were asked to report indicators that represent the benefit of the re-accreditation (see Additional file 1). However, it appeared that the connection between a benefit and the re-accreditation was difficult to evaluate, i.e., all interviewees reported great difficulties in saying whether a benefit could be directly associated to the re-accreditation and its activities. To evaluate whether there is a link between an identified benefit and the re-accreditation, the accountability criteria by Chassin et al. [19] can be used, which are i) research-robust evidence, ii) accuracy, iii) proximity, and iv) no adverse effect. Mumford et al. [4] added specificity as a fifth criterion to determine if the effects of accreditation can be isolated from other safety and quality measures. Despite the existence of such criteria, Mumford et al. [14] demonstrated the complexity of the identification of suitable indicators in their study, which we also experienced here (see Additional file 1 for how we incorporated the criteria in the interview guideline). For this reason, we asked the project leaders to estimate, based on their experience and knowledge, whether benefits occurred due to the re-accreditation. The same holds for the costs that had to be estimated.

As opposed to Mumford et al. [4], this study used one semi-structured interview guideline containing questions on costs and benefits, instead of several assessment tools. The questions in the semi-structured interview guideline were developed based on the questions from the assessment tools by Mumford et al. [4]. As a full qualitative analysis of the interviews was beyond the scope of this study, an approach as the one proposed by Halcomb and Davidson [20] was used. Field notes were taken in addition to audio recording. Key information was filtered out including standard-specific costs and working hours/days as well as information on indicators and the description of benefits.

Step 3: Quantification of costs and benefits

The quantification of costs was relatively straightforward. Either costs were already expressed in monetary units, or, in the case of working days, they were multiplied by a daily rate. The quantification of benefits on the other hand was significantly more complex. Most importantly, benefits of re-accreditation were typically not directly appraisable in monetary units. If benefits occur in non-monetary terms (due to non-market goods or services), a monetary value may be estimated [21]. The most common approaches include revealed and stated preference methods [21]. Due to the specific questions on re-accreditation in the KSGR and the limited resources available, this was not feasible in the present study. Instead, where quantification of the benefits was not possible, their description was considered to be the outcome. In addition, for two standards (QM and CIRS) a new way to quantify the benefits was developed. Together with the project leaders, a quantification grid was set up (see Additional file 2). The identified benefits from the interview were sent to the nine members of the quality commission to let them validate and rate the listed benefits. Given this approach, we could assign a numerical value to these benefits, which we call the Expert-Rated Benefit (ERB) score. While of course not equivalent to the approaches mentioned above that can yield monetary values of benefits, this method helped us to quantify the benefits in a standardized way.

Results

Identification and quantification of costs

The costs were collected for the whole accreditation project. Identified costs of the re-accreditation include costs for preparation and undergoing of the accreditation for all eight standards. Table 1 lists the costs in the interviews and the estimation by the quality manager or project leaders. Costs were separated by internal and external. External costs refer to expenses and payments made specifically in connection with the re-accreditation (e.g., accreditation fees). Internal costs include the amount of working days that employees spent for re-accreditation projects. Concerning monetization, external costs were already expressed in monetary units (CHF). Working days were monetized by multiplying each day by CHF 800 (according to the hospital’s budget guideline).

Table 1 Total identified costs regarding the sanaCERT re-accreditation (in CHF^a)

The identified costs amount to CHF 873'319 in total. Actual costs may slightly deviate from that because some costs had to be estimated. When differentiating between external and internal costs, external costs amount to CHF 154'151 whereas internal costs amount to CHF 719'168. The amount of total costs shows in fact a substantial investment. However, it is important to differentiate between internal and external costs. We can see that internal costs represent approximately 82 % of the total re-accreditation costs. Relative to the total amount of hospital expenses in 2013 (CHF 298'866'000), this amounts to only about 0.1 % annualized over three years. Other studies found slightly higher but still similar proportions of accreditation costs in total expenses of 0.3 % [22] and 1.7 % [12].

Considering the standards “QM” and “surgery” the costs in these standards appear to be higher compared to the other standards. The reason for this can mainly be found in personal intensive projects being carried out in these standards. For example, the standard “surgery” was involved in a nationwide project called “safe surgery”. This did require a lot of additional working hours.

Identification and quantification of benefits

The identification of quality indicators would have been a key requirement for quantifying the benefits of re-accreditation. The project leaders were not able to report suitable indicators that could be used to quantify the benefits. This was because (numeric) indicators and/or reliable data were missing. The project leaders were able to name topics, projects and services that improved in accordance with the re-accreditation. Consequently, it was only possible to report the benefits in a descriptive way rather than to quantify the data, with the exception of QM and CIRS, where we could quantify the benefits using our new validation tool. In the following, we will describe the results regarding what the project leaders considered to be the benefits of the re-accreditation.

When we first look at the nutrition standard, it reveals a controversial picture. According to the project leader, several benefits with additional profits of about CHF 1.5 Mio. have recently arisen due to different projects. These profits mainly arose due to the coding of malnutrition as a side diagnosis in the national reimbursement system. Some of the benefits of the nutrition standard were also possibly the result of projects realized in line with the first accreditation in 2011. Since this study focuses on the present re-accreditation cycle, and a direct link to the re-accreditation could not be established for these benefits and profits, they are not considered for a more detailed analysis here.

Next, we examine the benefits of the surgery standard. Table 2 lists the identified benefits from this standard. It would have been feasible to collect data on specific indicators (e.g., the number of hits on the surgery manual, a survey on employee’s satisfaction with projects, or the implementation of the surgery checklist). However, almost no data is available from prior to the accreditation and hence no before/after comparison is possible. Nonetheless, the described benefits represent important improvements that are valuable to the department and the hospital in general. For example, a better reporting of data on complications gives the experts the possibility to learn more about the underlying issues. Concerning the surgery manual, most of its documents and guidelines are not older than three years anymore. This indicates that the manual is up to date and used more frequently. Juul et al. [23] come to a similar finding, namely that the accreditation significantly increases both the quality and availability of clinical guidelines.

Table 2 Identified benefits in the Q-Standard surgery

With regard to the standards QM and CIRS, eight different benefits were identified in the interviews. Table 3 lists these benefits in the second column and also shows the outcome of the benefit rating activity by the members of the quality commission. Each expert rated every of the eight benefits according to two dimensions: First, the estimation of benefits on a scale form 0–3 (where 0 means no benefit, and 3 means enormous benefit), and secondly, the penetration in the hospital on a scale from 1–4 (where 1 means penetration in one division, and 4 means penetration througout the hospital). The results from the nine individual estimations by the experts for the two categories were averaged, resulting in the estimated benefits reported in Table 3, column 3, and the expected penetration reported in column 4. In a next step, these two numbers were multiplied (estimated benefit times expected penetration), which yields the ERB score for each benefit (column 5).

Table 3 Non-quantifiable benefit concerning the Q-Standards Quality Management and CIRS

The ERB score makes it possible to quantify and to compare the relative importance of the different benefits. According to our validation tool, the benefits “complaint management” and “quality-/safety culture” were rated the highest (7.5 and 10.3 points, respectively, out of 12 points at maximum). This indicates that those two benefits were evaluated as most important regarding the standards QM and CIRS by the experts. The order of the other benefits is: [3] fulfillment of the hospital vision, [4] improved image upon stakeholders (patients, suppliers etc.), [5] improved image in policy, [6] quality dashboard, [7] preparation of centralized quality documents, and [8] avoidance of liability cases.

The average ERB score over the eight identified benefits gives a value of 5.24. Assuming a conservative uniform distribution for the estimated benefit ratings and the expected penetration ratings with a correlation of 0.77 (which is the correlation found in the actual responses) we obtain a theoretical mean ERB score of 4.62, and thus the re-accreditation achieves an above-average rating by the experts in the selected benefits.

Finally, it was possible to identify a general benefit of the re-accreditation, namely a cultural benefit. This means that the presence of the re-accreditation awards the hospital a good reputation, at least temporarily. As hospital accreditation seems to be a general trend and something that the public possibly asks for, being accredited may itself be seen as a benefit. In this regard, Grepperud [2] described a possible shift in demand in favor of accredited hospitals if individuals value accredited hospitals more than non-accredited ones. This may result in a higher market share of accredited hospitals, which Grepperud [2] described as benefit. Furthermore, feeling the pressure of being audited may accelerate the process and implementation of some projects (e.g., the nutrition concept).

Discussion

The total identified costs of CHF 873'319 are a substantial investment, which gives immediate reason to question the decision to re-accredit the hospital. However, the number becomes relativized since the major part of the costs are internal amounting to CHF 719'168. For some of these internal costs, it may be questioned whether they exclusively arose because of the re-accreditation. One might conjecture that at least some of the working days and part of the expenses would have arisen also in the absence of the re-accreditation. In addition, putting the costs in relation to the hospital’s total expenses, they account for only about 0.1 %, which is even lower than the numbers found in related studies.

Hospital experts estimated a variety benefits in the studied standards of the KSGR, as a result of the re-accreditation. These include overall benefits, such as a cultural benefit, and more specific benefits like the improved capturing of complications in the surgery department. The quantified benefits of the standards QM and CIRS were quantified using a newly developed validation tool that assigns numerical values to the benefits. This can be seen as a step towards the quantification of descriptive benefits. According to this tool, the complaint management and quality or safety culture appear to be the most valued benefits. The average ERB score points to an overall beneficial effect of the re-accreditation and reinforces the use of accreditation as a quality assurance instrument.

The present study provides two major findings. First, it identifies the total costs and the benefits of the studied areas that arise from the re-accreditation of an acute care hospital in Switzerland. Secondly, it confirms the complexity of hospital accreditation as a subject matter and especially the problems involved in quantifying the benefits for further economic evaluation. This second aspect in particular may be the reason for the lack of relevant literature in the field of economic evaluation of hospital accreditation.

Due to the difficulty of quantifying and monetizing the benefits it was impossible to carry out a formal CBA, as designated by the SIQNS framework. Considering the type of analysis, our study probably comes closer to a CUA (in terms of economic evaluation methodology) due to the indirect valuation of the benefits of the standards QM and CIRS. Our results highlight that an economic evaluation of hospital accreditation is not impossible, though hard to carry out. Once proper data is available that allows for a quantification and monetization of benefits, it seems reasonable to conduct a formal CBA. A prospective view may help in the future to assure that quantifiable data is available. Maybe complementary to traditional approaches of valuing program benefits, the benefit identification procedure suggested here can be used to assess the benefits in absence of valid indicators and data. Thus, future studies may take place in a setting that allows for a more comprehensive view on assessing the benefits of (re-) accreditation. Finally, when looking at the decision to accredit, other aspects may be considered as well. To this end, Grepperud [2] suggested to incorporate how decisions depend on competition, the regulatory environment and the presence of alternative methods for evaluating hospital performance.

Our study also has several limitations. It needs to be acknowledged that both the benefit identification and quantification were mainly based on subjective estimates and thus are prone to bias. In addition, the project leaders faced an extremely difficult task by saying whether the respective benefits were clearly related to the re-accreditation, even though they were given some assistance with the accountability criteria in the interview guideline. Moreover, not all of the accountability criteria could be applied and in the end, the decision was mainly based on subjective estimates of the experts. Consequently, it was not possible to carry out a detailed analysis for the identification of possible indicators as proposed by Mumford et al. [4]. Only four out of eight standards were included in the study, which likely does not represent the full impact of the re-accreditation. By considering all eight standards, the total benefit of the re-accreditation may be higher. Concerning the costs, some of the internal costs needed to be estimated, which in fact may not represent the actual costs spent for the re-accreditation. However, in most cases, the costs and amount of working days are well documented and estimation should be sufficiently precise.

Conclusion

This study assessed the costs and benefits of re-accreditation of an acute-care hospital in Switzerland. Due to the lack of suitable indicators and data, a formal CBA could not be carried out. However, we suggest a novel validation tool that helps to quantify the descriptive benefits and that sets a basis for further research. The interpretation of the cost side of re-accreditation must consider the difference between internal and external costs to get a precise picture as some costs might have arisen even in the absence of the re-accreditation. The quantification of benefits is a difficult task. Nonetheless, this study provides a new approach to quantify the benefits in the given setting. After all, the study confirms the complexity and difficulty of economic evaluation of hospital accreditation. For this reason, a prospective and more comprehensive setting seems to be more feasible for future studies.