Open Access

Developing and testing quality indicators for the Thai Quality and Outcomes Framework

Safety in Health20173:14

https://doi.org/10.1186/s40886-017-0065-6

Received: 11 September 2017

Accepted: 30 October 2017

Published: 13 November 2017

Abstract

Background

Primary care serves as an entry point in the Thai health care system. Whilst effective interventions are provided in the primary care setting, the quality of the services have not been measured or tracked. A number of initiatives were undertaken to improve primary health care quality including the use of financial incentives to reward adherence to performance indicators. However, there were concerns that the current quality indicators had not been developed in a systematic, participatory, and evidence-based manner. Therefore, this study aims to develop new quality indicators for use in subsequent iterations of the program.

Methods

The development of indicators follows a well-designed approach. Reviews of existing documents as well as secondary data analyses were performed and presented to key stakeholders. Disease areas were then prioritised. Recommendations from the Thai clinical practice guidelines on the prioritised areas were then used to formulate statements and templates for each indicator. Finally, the indicators developed were piloted for 3 months in 28 primary care units across the country.

Results

Indicators related to care for diabetes and hypertension, maternal and child health, and rational use of antibiotics received high acceptability, and information was available and collectable in the current administrative database. However, there were problems in implementing indicators for managing cardiovascular risk, care for bedridden patients, and asthma and COPD.

Conclusions

The development of quality indicators using a guideline-based approach is a useful way of generating evidence to support the effective implementation of a program. Indicator piloting is recommended prior to introducing indicators in the health system.

Keywords

IndicatorsPrimary careQuality of carePay for performance

Background

Primary care in Thailand serves as the entry point in the health care system. Beneficiaries of the Universal Coverage Scheme (UCS), the publicly financed health insurance program, cover 75% of the Thai population, who are required to register at the contracting unit for primary care (CUP) in their residential area. In general, a CUP includes one district hospital and several health-promoting hospitals or primary care units (PCUs). District hospitals offer both primary and secondary care services whilst health-promoting hospitals provide ambulatory care and community services. These are staffed by nurse practitioners and public health officers [1].

A key feature of the UC scheme is the Thai Quality and Outcomes Framework (Thai QOF). This pay-for-performance scheme was introduced in 2013 to address variations in the quality and accessibility of primary care. Health care providers are incentivised to improve primary care quality in key predetermined areas expressed as a series of quality indicators. Achievement against each indicator is calculated on an annual basis using national administrative records and given a point value. The total points obtained are then converted into a financial value, which is allocated to the CUPs [2].

Quality indicators are an essential component of this program as they influence both provider effort and subsequent funding allocation [3, 4]. However, early evaluation of the program identified that the quality measures in use were perceived to have poor validity which negatively impacted upon the acceptability of the scheme, with resulting variation in uptake and financing at the local level [5]. In response, an independent research unit in the Ministry of Health-the Health Intervention and Technology Assessment Program (HITAP) was commissioned to develop and pilot potential quality indicators to be implemented in fiscal year 2017. This was done in collaboration with the National Institute for Health and Care Excellence (NICE) National Collaborating Centre for Indicator Development, University of Birmingham, and former NICE International.

Two broad approaches have been used to develop quality indicators: a deductive approach and an inductive approach [6]. A study conducted by Stelfox and Straus found that the majority of quality indicators were developed using a deductive approach [6, 7]. This approach suggests that the quality indicators should be derived from scientific evidence related to important quality-of-care concepts. Studies in the UK suggest that the key attributes of good quality indicators are reliability, validity, feasibility, acceptability, being attributable, and sensitivity to change. Reliability is the extent to which the performance of an indicator is consistently assessed. Validity represents the extent to which evaluators are measuring what they intend to measure. Feasibility considers the extent to which the necessary data are available and collectable. Acceptability refers to whether the indicator is acceptable and fair as viewed by health professionals; it also covers any unintended consequences resulting from the introduction of an indicator. Being attributable considers whether the indicator is in the control of health professionals. Finally, sensitivity to change refers to the ability of an indicator to detect changes in care quality [8, 9].

Since more than 3% of the primary care budget is being spent on the QOF program [10], piloting the indicators is desired in order to ensure efficient health care spending. In addition, indicator piloting provides an opportunity to prove the value and reveal practical issues of the developed indicators before spending a significant amount of time or money on actual implementation [8, 9, 11].

Methods

The development of quality indicators was carried out in two stages: indicator development (May–November 2015) and indicator testing (December 2015–February 2016). The process was overseen by a steering committee, chaired by the Director of the Healthcare Accreditation Institute. The steering committee provided strategic direction including the prioritisation of disease areas, potential indicators in each area, the approval of the indicator testing protocol, and the recommendation about the final set of indicators.

Indicator development

The overall process of quality indicator development and testing is described in Fig. 1. The process began with prioritising areas/issues that needed quality improvement. This was followed by developing the initial set of indicators based on clinical practice guidelines and testing them.
Fig. 1

Flow diagram of the indicator development process

Prioritisation of areas requiring quality improvement

Prioritisation of clinical areas requiring quality improvement was guided by five criteria agreed by the steering committee and was developed with reference to international literature on priority setting. Available evidence on the principles or criteria for setting priorities in primary care services was reviewed. Literature in the USA showed that different principles (e.g. safe, person-centred care, affordability) were considered in setting the national strategy for quality improvement [12]. In Sweden, the criteria included severity of the health condition, patient benefit, and cost-effectiveness [13]. Korea used the burden of condition, seriousness of the quality problem, interest and demand of society, acceptability, and feasibility as criteria for setting national priorities for quality assessment of health care services [14]. Based on this review, five criteria were selected for priority setting in Thailand: (1) burden of condition, (2) variation in quality of care, (3) availability of clinical practice guideline(s), (4) feasibility of quality assessment, and (5) the extent to which the area is in line with national policy. These criteria were ratified during two stakeholder meetings which included policy-makers from the Ministry of Public Health (MOPH) and National Health Security Office (NHSO), academics, regional health managers, and primary health care workers.

Prior to these stakeholder meetings, the research team developed a shortlist of eight areas thought to be suitable for quality improvement because these problems had high levels of burden of disease and variation in quality of care. Stakeholders were then asked to discuss these and to individually select the three areas they perceived as being most suitable for quality measure development. The five most endorsed areas from both groups were presented to the steering committee, which made the final determination of priority areas. These included chronic diseases identification and management (hypertension, diabetes, and cardiovascular diseases), maternal and child health, care of the bedridden patients, rational use of antibiotics, and asthma and chronic obstructive pulmonary disease (COPD).

Selection of clinical practice guideline(s)

Recommendations from clinical practice guidelines were used as a basis for formulating the quality indicators. Available Thai clinical practice guidelines in the selected clinical topics were assessed using the AGREE appraisal tool [15]. The research team also consulted experts in each area to understand the use of guidelines in primary care units and identify related service delivery issues. Guidelines deemed appropriate were subsequently used to formulate the indicators.

Drafting the initial set of indicators

Twenty-six indicators were developed by the research team and discussed with the steering committee. Seven indicators were excluded because they did not add value or they were anticipated to be difficult to assess through the existing information system. However, five new indicators were suggested by the steering committee including the control of blood pressure and blood glucose levels, referral of pregnant women with high risk of pregnancy, visits of a family care team to bedridden patients, and admission rates of patients with an exacerbation of asthma to the emergency department. Altogether, 24 key indicators were recommended for piloting.

The indicators were then assessed with a focus group for content validity in terms of clarity of wording and necessity, defined as whether primary care units adhering to the indicator provided higher quality care/service than those who were not doing so [8]. The group included three experts in selected disease areas, two NHSO staff, four representatives from provincial and district health offices in different regions, and two health care workers from primary care units. The indicator template and questions were sent to participants prior to the meeting. Participants discussed each indicator in turn, and comments and suggestions were reviewed by HITAP. Changes to indicator wording were then made as a result of the discussion.

Indicator testing

Indicators were tested through a 3-month pilot in a purposive sample of three geographically diverse regions in the south, north-east, and central parts of Thailand. These regions were identified based upon the willingness of the provincial health offices to participate in the pilot. The provincial health officers in each province were asked to select three CUPs using the following criteria: population size according to the UC population and location of the CUP (a mix between rural and urban). Then, three primary care units (PCUs) under those selected CUPs were selected based on their willingness to participate in the study as well as the type of PCU (one PCU managed by the hospital and two health-promoting hospitals). The recruitment process of primary care units is illustrated in Fig. 2.
Fig. 2

Recruitment process of primary care units for testing quality indicators. *CUP is abbreviated from Contracting Unit for Primary Care. **PCU is abbreviated from Primary Care Unit

Participating PCUs were informed about the indicators and also given a study handbook and details of the electronic data extraction. The study handbook provided all of the details regarding the indicators, aim of the pilot, timelines, responsibility of PCUs during the pilot, and the process of indicator evaluation after piloting.

Data collection and analysis

Mixed methodologies were used to collect and analyse data including self-administered questionnaires, interviews, and data extraction of patient’s medical records.

Self-administered questionnaires

For every pilot site, descriptive information was collected on population characteristics, opening hours, and staffing using a self-administered structured questionnaire. Descriptive statistics were used to describe and summarise the data. In addition, all staff involved in the pilot were asked to complete a workload questionnaire at the end of the pilot. The questionnaire included date and time of activity and any equipment used to provide services. The questionnaire aimed to quantify the time (in minutes) that was spent on each activity including work with patients and administrative work.

Semi-structured interviews

Semi-structured interviews with open-ended questions (see Appendix 1) were undertaken with PCU staff to obtain information regarding acceptability, feasibility, potential barriers, and unintended consequences of the introduction of the indicators. Interviews were coded and analysed thematically. Regarding acceptability, participants were asked whether they would accept the indicators for the Thai QOF program. The acceptability rate for each indicator was grouped.

Administrative database

QOF achievement was calculated using patient-level information held in a national database managed by the MOPH. This database contains information about the patient, their use of ambulatory services, and inpatient services of health facilities under the federal government [16]. For an indicator to be feasible, the national database must hold information about the specified care activity. The availability of information was categorised based on the specified care activity at the individual indicator level into three groups: (1) necessary information is available in the national database and extraction rules can be applied; (2) necessary information might be available in the national database but staff did not enter the necessary data; and (3) necessary information is not collected in the national database.

Post-pilot selection of potential indicators for the Thai QOF program

Two consultation meetings were organised in March 2016. The aims of these meetings were for the research team to present the preliminary findings of the pilot. The first meeting involved 15 key stakeholders at the national level including policy-makers from the MOPH and researchers from academic institutes, as well as representatives from the NHSO and its regional offices. At the second meeting, 14 participants attended including representatives from provincial and district health offices and health facilities. Comments and suggestions on the pilot findings were summarised and presented to the steering committee.

Results

Assessment of clinical practice guidelines

The result of guideline assessment is provided in Table 1. These guidelines deemed appropriate to use for indicator development.
Table 1

The score of the quality of clinical practice guidelines for developing quality indicators

Guideline

Criteriaa

 

The overall quality of the guideline development

The overall quality of the guideline presentation

The completeness of reporting

The overall quality of the guideline recommendations

The overall quality of the guideline

Hypertension

5

5

1

7

5

Diabetes

6

6

1

7

6

Cardiovascular diseases

4

4

1

6

5

Rationale use of antibiotics

5

6

1

7

5

Asthma

6

6

1

7

6

COPDb

6

6

1

7

6

aThe response is on a 7-point scale ranging from the lowest quality (1) to the highest quality (7)

bCOPD is abbreviated from chronic obstructive pulmonary disease

Indicator development

In total, 24 indicators were suggested for piloting. The list of indicators is presented in Table 2.
Table 2

Developed indicators for the QOF program

Indicator code

Indicator statement

HT1 (PCU)

Percentage of individuals aged 35 and above who received a screening for hypertension in the last 3 years

HT2 (PCU)

Percentage of individuals with blood pressure 140/90 mmHg and above who were referred to a physician within 1 month

HT3 (PCU)

Percentage of hypertensive patients who could control their blood pressure

DM1 (PCU)

Percentage of individuals aged 35 and above who received a screening for DM using random or fasting capillary blood glucose in the last 3 years

DM2 (PCU)

Percentage of individuals with capillary blood glucose of 126 mg/dl and above who received fasting venous plasma glucose test within 1 month

DM3 (PCU)

Percentage of DM patients who could control their blood glucose

CVD1 (PCU)

Percentage of individuals aged 35–70 who receive CVD risk assessment using the Thai CV risk score

MCH1 (PCU)

Percentage of pregnant women who received antenatal care (ANC) for the first time before 12 weeks

MCH2 (PCU)

Percentage of pregnant women with hypertension who were referred to a physician

MCH3 (CUP)

Percentage of pregnant women with anaemia in the 1st trimester whose Hct are in normal range in the third trimester

MCH4 (CUP)

Percentage of postpartum women with anaemia during pregnancy whose Hct are in normal range at 3 months after delivery

MCH5 (PCU)

Percentages of full-term infants aged between 0 and 1 year whose weight for age or weight for height or height for age falls below the 5th percentile or 95th percentile and above who are referred to doctors.

BR1 (PCU)

PCUs can produce a register of bedridden patients

BR2 (PCU)

Percentage of bedridden patients who were visited by a family care team

RUA1 (PCU)

Percentage of antibiotics prescription for patients diagnosed with upper respiratory tract infections (URI)

RUA2 (PCU)

Percentage of antibiotic prescription for patients diagnosed with acute gastroenteritis (AGE)

ASTHMA1 (PCU)

PCUs can produce a register of patients with asthma

ASTHMA2 (PCU)

Percentage of asthma patients who received health education and counselling

ASTHMA3 (PCU)

Percentage of asthma patients who were examined with a peak expiratory flow meter

ASTHMA4 (PCU)

Percentage of asthma patients who are assessed by an Asthma Control Test

ASTHMA5 (PCU)

Percentage of asthma patients who could not control asthmatic symptoms and were referred to a physician

ASTHMA6 (PCU)

Percentage of asthma patients with asthmatic exacerbation that were admitted to the emergency department

COPD1 (PCU)

PCUs can produce a register of patients with COPD

COPD2 (PCU)

Percentage of COPD patients who are currently smoking receiving advice on smoking cessation

Indicator testing

Twenty-eight PCUs participated in this pilot study including 5 PCUs with less than 3000 people, 13 PCUs with 3001–7000 people, 5 PCUs with 7001–10,000 people, and 5 PCUs with over 10,000 people. Staffing was different according to the types and size of PCUs. PCUs under district hospitals were staffed by approximately 2–4 physicians, 1–3 nurses, 1–2 dentists, 2–3 public health officers, and 1 staff who managed data. Health-promoting hospitals were staffed by 1–2 nurse practitioners, 1–2 nurses, and 1–4 public health officers.

Acceptability and implementation

Informants included 25 PCU directors, all 91 health care workers who deliver services related to the tested indicators, and 15 data managers during the pilot. The acceptability of indicators is illustrated in Table 3.
Table 3

Availability and acceptability of quality indicators

Acceptability

Availabilitya

1. Necessary information is available

2. Necessary information might be available

3. Necessary information is not collected

1. ≥ 70%

HT1, HT2, HT3, DM1, DM3, MCH2, MCH5

DM2, MCH3, Asthma1, COPD1, COPD2

BR1, BR2, Asthma3, Asthma4, Asthma5, CVD1

2. 60–69%

MCH1, RUA1, RUA2

  

3. 50–59%

 

Asthma2, Asthma6

 

4. < 50%

 

MCH4

 

a(1) Necessary information is available in the national database and extraction rules could be applied; (2) necessary information might be available in the national database but staff did not enter the necessary data; and (3) necessary information is not collected in the national database

Indicators related to care for diabetes, hypertension, maternal and child health, and bedridden patients were highly acceptable. The reasons for this were that the indicators were perceived to be beneficial both for people and health providers; PCUs were perceived to have the capacity to provide services in terms of staffing, technology, and skills; non-communicable diseases (NCDs) were priorities for quality improvement due to the associated burden; and indicators were viewed as a good guide for health providers to know the priority areas of health services and how good quality services should be provided.

Some of the results are illustrated in the comments from participants

“The indicators on hypertension and diabetes screening should be included in the QOF program because it provides information on the standard of the care for hypertension and diabetes - which are important health problems in our area. If we are able to screen more people, we might detect more patients in the early stages of the diseases and this will result in better prognosis. Previously, when we did not have such indicators, people who are in the early stages of the diseases were hidden in the community and did not get access to the care”—nurse practitioner in a health centre.

Implementation concerns related to some indicators raised by participants included the lack of support systems from other agencies (e.g. training, databases, feedback system), lack of knowledge of heath care workers to provide services, lack of equipment, lack of human resources, and some indicator criteria that were out of the providers’ control such as lifestyle modification-related issues. For example, the indicator on screening for cardiovascular diseases using the Thai global risk score was really appreciated by policy-makers and was perceived as necessary by experts; however, health care workers did not accept this indicator because they did not have the capacity to carry out related activities in terms of equipment, knowledge, and skills. Moreover, data related to this indicator is not currently available in the routine database.

Some of the results are illustrated in the comments from participants

“Our health center has just been informed about the policy of cardiovascular diseases screening with the Thai Cardiovascular (CV) risk score. We have not prepared for its implementation yet. We have not trained our staff to be able to delivery this service, or provide them with the needed materials. If we need to screen the general population aged between 35-70 years, it will increase our workload a lot. We are not ready for this indicator”—nurse practitioner in a health centre.

Data availability

Examination of the care activities collected as part of the national health database revealed some issues with data availability. These results are presented in Table 3. When considering the completeness of data, it showed that the rates of missing data were exceptionally high for variables related to referral services and smoking status (51–98%). In addition, we found that staff incorrectly entered the International Classification of Diseases 10th Revision (ICD-10) code for hypertension during pregnancy by entering the code for hypertension in the general population.

Workload

The analysis of workload information revealed that most of the activities related to the piloted indicators were carried out by practice nurses. Figure 3 shows that health care workers spent the most time on care for bedridden patients in the community, followed by registering bedridden patients and providing first antenatal care for pregnant women. Assessing lung capacity by peak flow expiratory meter and providing asthmatic control test took the least time.
Fig. 3

Workload associated with developed indicators

Overall recommendations

Based on the summarised information gathered from the pilot and two consultation meetings, recommendations were made on which indicators the NHSO should implement in the QOF program. Ten indicators were recommended for implementation in the next QOF program because the indicators were accepted by health care workers, feasible for implementation in the primary care setting, and data related to these indicators were available in routine databases. The list of recommended indicators is presented in Table 4.
Table 4

Selected indicators for the future QOF program

Indicator code

Indicator statement

HT1 (PCU)

Percentage of individuals aged 35 and above who received a screening for hypertension in the last 3 years

HT2 (PCU)

Percentage of individuals with blood pressure 140/90 mmHg and above who were referred to a physician within 1 month

HT3 (PCU)

Percentage of hypertensive patients who could control their blood pressure

DM1 (PCU)

Percentage of individuals aged 35 and above who received a screening for DM using random or fasting capillary blood glucose in the last 3 years

DM2 (PCU)

Percentage of individuals with capillary blood glucose of 126 mg/dl and above who received fasting venous plasma glucose test within 1 month

DM3 (PCU)

Percentage of DM patients who could control their blood glucose

MCH2 (PCU)

Percentage of pregnant women with hypertension who were referred to a physician

MCH5 (PCU)

Percentages of full-term infants aged between 0 and 1 years whose weight for age or weight for height or height for age falls below the 5th percentile or 95th percentile and above who are referred to doctors within 12 months

RUA1 (PCU)

Percentage of antibiotics prescription for patients diagnosed with upper respiratory tract infections (URI)

RUA2 (PCU)

Percentage of antibiotic prescription for patients diagnosed with acute gastroenteritis (AGE)

ASTHMA1 (PCU)

PCUs can produce a register of patients with asthma

ASTHMA2 (PCU)

Percentage of asthma patients who received health education and counselling

Discussion

This paper describes the process of quality indicator development and piloting for the Quality and Outcomes Framework in Thailand. To our knowledge, this is the first time that Thailand has developed quality indicators for primary care following systematic and transparent processes, incorporating evidence for prioritising areas, and using recommendations in clinical guidelines to ensure that quality indicators attribute to better health outcomes. In addition, the developed indicators were assessed using real-world data. Indicator piloting allowed us to identify a number of implementation issues and concerns of health care providers specific to the pilot indicators. It also enabled us to explore data availability in order to assess technical feasibility of the indicators.

This study confirms that piloting should be a prerequisite for policy-makers before they introduce indicators in the health system [9]. Our study showed that although indicators were systematically developed according to a well-designed approach, their implementation in practice was not straightforward. The indicator on screening for cardiovascular diseases using the Thai global risk score was a good example. As a result, this indicator was not recommended unless health care workers were trained, equipment was provided, and the national health database was amended to capture the care activity.

Another valuable aspect of testing indicators is that it allows problems to be identified and solutions to be prepared prior to implementation. For example, the ICD-10 code for hypertension during pregnancy was incorrectly entered. Therefore, we recommended that the NHSO provides guidance for PCUs on how data related to each indicator should be entered. Similarly, we found a high degree of missing data for referral indicators. By verifying this finding with some PCUs, we found that the missing data was not a reflection of the care not having been delivered but the data was not entered in the database. Applying a financial incentive could result in better, more consistent data entry. We confirmed that the availability and comprehensiveness of clinical data are crucial for the P4P program as suggested by other studies [8]. The health information system was not created specifically for the QOF program, resulting in incompleteness of data to assess performance based on some indicators. We therefore recommend the MOPH to improve the national database to be more useful for future QOF and other initiatives on primary care. Using administrative data for measuring quality of care could provide advantages because these secondary data are readily available and are relatively inexpensive. The USA used hospital claims data and out-patient claims data to measure the quality of medical care provided to Medicare beneficiaries [17].

The process of indicator development and piloting as described in this study might be applied in other resource-constrained settings. This work followed a systematic approach to indicator development and testing, drawing upon protocols used to support indicator development in the UK and elsewhere [8, 18] and adapting culturally to ensure sensitivity and feasibility in the Thai context. Some attributes (cost-effectiveness, sensitivity to change, and reliability) were not feasible for assessment in our context given the high PCU workload and short pilot time frame. However, we believe that our pilot provides enough information for policy-makers as the tested key attributes are similar to key attributes described in the QUALIFY protocol for testing quality indicators in Germany [19].

This study provides recommendations on 10 indicators that should be implemented for the fiscal year 2017. It also highlights 14 indicators that should be postponed until the health database is ready or health care staff is able to provide services. In our context where indicators are part of our health care system, this is the best available evidence so far that can be used to convince organisations about the value of indicator piloting. Policy-makers and program implementers at both the national and international levels can benefit from this study for ensuring the development of acceptable, feasible, and evidence-based indicators.

The study has several limitations. Firstly, we did not assess the representativeness of stakeholders participating in the prioritisation process. Although we invited key policy-makers from different departments in the MOPH, not all invited departments joined the meeting. We were also unable to include patient representatives and lay people in this process despite knowing their crucial roles. Secondly, we considered adopting a RAND/UCLA Appropriateness Method [20] to rate the clarity and necessity of each indicator but ended up with a focus group discussion as participants did not rate the indicators prior to the meeting and did not agree to rate it during the meeting. This unsuccessful activity might be explained by the following: (1) the number of indicators to be assessed was too high for participants to be able to rate in the given timeframe; (2) the time given for participants (15 days) was too short; and (3) participants were not familiar with this method and might not appreciate its benefits. Thirdly, health providers failed to keep workload diaries because they found it too burdensome in addition to their usual work. Therefore, workload was measured at the end of the pilot. The reported workload might over- or underestimate the real value as it was measured at a single point in time and did not account for variations across the piloting period. Also, the workload is calculated based on PCUs that provided these activities, so a low workload might be due to non-participation. Finally, due to tight deadlines in the overall process, the piloting time of 3 months might be too short to assess indicators comprehensively compared to 6 months in the UK QOF [8] or 12 months in the US HEDIS [18]. For example, the analysis of sensitivity to change was not performed as data after piloting could not be extracted from the database. Similarly, implementation issues or unintended consequences might not have been captured as staff would need more time to engage with the indicators.

Conclusions

This systematic, evidence-based indicator development for the QOF or other pay-per-performance initiatives is necessary and feasible in resource-constrained settings. The study also confirmed that indicator piloting is a prerequisite for policy-makers prior to introducing indicators in the health system since it ensures acceptability, feasibility, relevance, and effectiveness of the indicators.

Abbreviations

COPD: 

Chronic obstructive pulmonary disease

CUP: 

Contracting unit for primary care

CV risk: 

Cardiovascular risk

HITAP: 

The Health Intervention and Technology Assessment Program

ICD-10: 

International Classification of Diseases 10th Revision

MOPH: 

Ministry of Public Health

NHSO: 

National Health Security Office

NICE: 

The National Institute for Health and Care Excellence

PCUs: 

Primary care units

Thai QOF: 

Thai Quality and Outcomes Framework

UCS: 

Universal Coverage Scheme

Declarations

Acknowledgements

The authors would like to express a deep gratitude and sincere appreciation to Dr. Anuwat Supachutikul for chairing the steering committee and the steering committee for their supervision; Dr. Thunyarat Anothaisintawee for her support and guidance during the development of indicators; and Rukmanee Butchon, Suthasinee Kamluang, Suteenoot Tangsatitkulchai, and Natthida Malathong for their assistance. In addition, we would like to thank Dr. Choochai Sornchumni, Dr. Jakkrit Ngowsiri, Ms. Bumrung Chalodech, and the staff at the NHSO for their guidance and assistance during the data collection. Additionally, we would like to express our special thanks to all informants for their contribution to this study.

HITAP is funded by the Thailand Research Fund under the senior research scholar on Health Technology Assessment (RTA59800011) and the Bureau of Health Policy and Strategy, Ministry of Public Health. HITAP’s international unit has been supported by the international Decision Support Initiative (funded by the Bill & Melinda Gates Foundation and the Department for International Development, UK) and the Rockefeller Foundation to provide technical assistance on health intervention and technology assessment for governments of low- and middle-income countries. The findings, interpretations, and conclusions expressed in this article do not necessarily reflect the views of the aforementioned funding agencies.

Funding

Financial support for this study was provided entirely by the Thai National Health Security Office, grant number I-59-3-01-358-02-2. The findings, interpretations, and conclusions expressed in this article do not necessarily reflect the views of the aforementioned funding agency.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on a reasonable request.

Authors’ contributions

RK collected the data, analysed and interpreted the findings, and was a major contributor in writing the manuscript. ST, YT, FC, RF, and PG supervised the overall process of the study and provided feedback and suggestions on the manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate

Ethical approval for this study was obtained from the Institute for the Development of Human Research Protection, Thailand (http://www.ihrp.or.th/ ), document number 1629/2558. All informants provided written consent before participating in the study. Moreover, data were analysed and presented anonymously.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Health Intervention and Technology Assessment Program (HITAP)
(2)
Institute of Global Health Innovation, Imperial College London
(3)
NICE National Collaborating Centre for Indicator Development, University of Birmingham
(4)
Warwick Medical School, University of Warwick

References

  1. Tangcharoensathien V, Limwattananon S, Patcharanarumol W, Thammatacharee J, Jongudomsuk P, Sirilak S. Achieving universal health coverage goals in Thailand: the vital role of strategic purchasing. Health Policy Plan. 2015;30(9):1152–61.View ArticlePubMedGoogle Scholar
  2. National Health Security Office: Guidance on QOF indicators in 2016. Nonthaburi: National Health Security Office; 2015.Google Scholar
  3. Donabedian A. The quality of care: how can it be assessed? JAMA. 1988;260(12):1743–8.View ArticlePubMedGoogle Scholar
  4. Chiu WT, Yang CM, Lin HW, Chu TB. Development and implementation of a nationwide health care quality indicator system in Taiwan. Int J Qual Health Care. 2007;19(1):21–8.View ArticlePubMedGoogle Scholar
  5. Khampang R, Tantivess S, Teerawattananon Y, et al. Pay-for-performance in resource-constrained settings: Lessons learned from Thailand’s Quality and Outcomes Framework. F1000Research. 2016;5:2700. doi:10.12688/f1000research.9897.1.
  6. Stelfox HT, Straus SE. Measuring quality of care: considering conceptual approaches to quality indicator development and evaluation. J Clin Epidemiol. 2013;66(12):1328–37.View ArticlePubMedGoogle Scholar
  7. Stelfox HT, Straus SE. Measuring quality of care: considering measurement frameworks and needs assessment to guide quality indicator development. J Clin Epidemiol. 2013;66(12):1320–7.View ArticlePubMedGoogle Scholar
  8. Campbell SM, Kontopantelis E, Hannon K, Burke M, Barber A, Lester HE. Framework and indicator testing protocol for developing and piloting quality indicators for the UK quality and outcomes framework. BMC Fam Pract. 2011;12:85.View ArticlePubMedPubMed CentralGoogle Scholar
  9. Lester H, Campbell S. Developing quality and outcomes framework (QOF) indicators and the concept of ‘QOFability’. Qual Prim Care. 2010;18(2):103–9.PubMedGoogle Scholar
  10. National Health Security Office. Estimated budget of the universal coverage scheme for fiscal year 2014 [http://www.nhso.go.th/frontend/NewsInformationDetail.aspx?newsid=NjM1]. Accessed 15 Jan 2017.Google Scholar
  11. Young GJ, Charns MP, Barbour GL. Quality improvement in the US Veterans Health Administration. Int J Qual Health Care. 1997;9(3):183–8.View ArticlePubMedGoogle Scholar
  12. National Strategy for Quality Improvement in Health Care [http://www.ahrq.gov/workingforquality/nqs/nqs2011annlrpt.htm.]. Accessed 15 Jan 2017.
  13. Arvidsson E. Priority setting and rationing in primary health care [doctoral thesis]. Sweden: Linköping University; 2013.Google Scholar
  14. Cho W, Lee S, Kang HY, Kang M. Setting national priorities for quality assessment of health care services in Korea. Int J Qual Health Care. 2005;17(2):157–65.View ArticlePubMedGoogle Scholar
  15. Appraisal of guidelines for research & evaluation II [www.agreetrust.org]. Accessed 15 Jan 2017.
  16. Wangkrapong P, Suvanprateeb T, Kusolnumpa S, Kusakunniran W, and Thongkanchorn K. Electronic health information standard for patient transfer based on CDA, focused on patient information in small-sized hospitals (JHCIS). In 2015 International Computer Science and Engineering Conference (ICSEC), Chiang Mai, 23-26 November 2015.Google Scholar
  17. Jencks SF, Cuerdon T, Burwen DR, Fleming B, Houck PM, Kussmaul AE, Nilasena DS, Ordin DL, Arday DR. Quality of medical care delivered to Medicare beneficiaries: a profile at state and national levels. JAMA. 2000;284(13):1670–6.View ArticlePubMedGoogle Scholar
  18. HEDIS. Technical specifications for physician measurement. Washington DC: NCQA; 2009.Google Scholar
  19. Reiter A, Fischer B, Kötting J, Geraedts M, H Jäckel W, Barlag H, Doebler K. QUALIFY: Instrument for the Assessment of Quality Indicators. [https://www.researchgate.net/publication/267256474_QUALIFY_Instrument_for_the_Assessment_of_Quality_Indicators]. Accessed 15 Jan 2017.
  20. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, JR LC. RAND/UCLA. Appropriateness. Method. User’s manual. Arlington: RAND; 2001.Google Scholar

Copyright

© The Author(s). 2017

Advertisement