Health Policy and Planning Advance Access published online on April 5, 2007
Health Policy and Planning, doi:10.1093/heapol/czm010
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Priority setting using multiple criteria: should a lung health programme be implemented in Nepal?
1Department of Public Health, Radboud University Medical Centre Nijmegen, Nijmegen, The Netherlands.
2Institute for Medical Technology Assessment, Erasmus Medical Centre Rotterdam, The Netherlands.
3Department of Social Medicine, Academic Medical Centre, University of Amsterdam, The Netherlands.
4Department of Health Policy and Management, Erasmus Medical Centre Rotterdam, The Netherlands.
5Department of Community Medicine and Family Health, Institute of Medicine, Tribhuvan University, Kathmandu, Nepal.
6Nepal Institute of Health Sciences, Kathmandu, Nepal.
* Corresponding author. Department of Public Health, Radboud University Medical Centre Nijmegen, Nijmegen, The Netherlands. Tel: +31 24 3613235/3119. Fax: +31 24 3619561. E-mail: r.baltussen{at}sg.umcn.nl
| Abstract |
|---|
|
|
|---|
Objectives To identify and weigh the various criteria for priority setting, and to assess whether a recently evaluated lung health programme in Nepal should be considered a priority in that country.
Methods Through a discrete choice experiment with 66 respondents in Nepal, the relative importance of several criteria for priority setting was determined. Subsequently, a set of interventions, including the lung health programme, was rank ordered on the basis of their overall performance on those criteria.
Results Priority interventions are those that target severe diseases, many beneficiaries and people of middle-age, have large individual health benefits, lead to poverty reduction and are very cost-effective. Certain interventions in tuberculosis control rank highest. The lung health programme ranks 13th out of 34 interventions.
Conclusion This explorative analysis suggests that the lung health programme is among the priorities in Nepal when taking into account a range of relevant criteria for priority setting. The multi-criteria approach can be an important step forward to rational priority setting in developing countries.
Key Words: Multi-criteria decision analysis, priority setting, rational, Nepal
KEY MESSAGES
|
| Introduction |
|---|
|
|
|---|
With the advent of many new initiatives to increase funding for health care in developing countries, such as the Global Fund to Fight HIV/AIDS, Tuberculosis and Malaria (Global Fund 2005
However, priority setting has often been implicit, based on decisions made in the past, or resulting from unintended outcomes of the various pressures on providers, government agencies, purchasers and patients (Ham 1997
; Martin and Singer 2000
; Segal and Chen 2001
). It is argued that only by using an explicit approach to priority setting can resources be directed to maximize the achievement of societal objectives in relation to health (Segal and Chen 2001
). The use of cost-effectiveness analysis has often been proposed as such an explicit approach, but many other criteria for priority setting have also been put forward, including medical (such as burden of disease) and non-medical criteria (such as age of target population) (Kapiriri et al. 2003
; Kapiriri and Norheim 2004
; Kapiriri et al. 2004
; Baltussen and Niessen 2006
). Recently, the development of a multi-criteria approach to priority settingor more generally labelled multi-criteria decision analysis (MCDA)has been identified as one of the important issues in health system research (Mills et al. 2004
).
In stark contrast with the near-absence of applications of MCDA to allocation decisions in health, is the widespread acceptance and routine use of MCDA in other disciplines, for example to structure remedial decisions at contaminated sites in environmental sciences (Linkov et al. 2004
). MCDA has also been applied in agricultural (Land and Plant Nutrition Management Service 2005), energy (Hirschberg et al. 2004
) and marketing (McDaniel et al. 2005) sciences. In those disciplines, MCDA has evolved as a response to the observed inability of people to effectively analyse multiple streams of dissimilar information. The analysis establishes preferences between intervention options by reference to an explicit set of criteria that the decision-making body has identified (National Economic Research Associates 2000
).
Following the experience of MCDA in those disciplines, we aim to develop a similar approach to prioritize intervention options in health. It has been argued that successful MCDA in health should: (1) involve quantitative rather than qualitative analysis since the priority-setting process involves many criteria and many interventions, and intuitive processing of this data can lead to unjustified conclusions; (2) cover a comprehensive set of criteria relevant for decision-making; (3) consider criteria simultaneously to allow trade-offs between various criteria; (4) establish the relative importance of criteria in a way that allows a rank ordering of a comprehensive set of interventions; and (5) be strongly embedded in the organizational context (Baltussen and Niessen 2006
). This paper explores the potential of discrete choice experiments (DCE) in this process, carried out alongside similar studies in other settings (Baltussen et al. 2006
). DCE provide one way of analysing and communicating preferences for different interventions to policy makers (Ryan and Gerard 2003
; Kjaer 2005
).
In an explorative analysis, the approach is used to support current policy making in Nepal on the implementation of the Practical Approach to Lung health (PAL) programme. The programme, initiated by the World Health Organization, introduces a set of guidelines on integrated case management of pneumonia, tuberculosis (TB), chronic obstructive pulmonary disease (COPD) and asthma (Scherpbier et al. 1998
; Nelson et al. 2004
; WHO 2004
; ten Asbroek et al. 2005
), and is intended to promote better lung care for school-aged children, youths and adults at first-level health facilities. In a pilot implementation in Nepal, PAL was found to be more costly but more effective than current treatment guidelines for lung disease in Nepal, and thereby relatively cost-effective (Shrestha et al. 2005
). The topic of discussion in Nepal is now whether PAL should be scaled up to other districts and eventually to the whole country. The nation-wide implementation of the intervention will require significant resources (Samir et al. 2005
), and thereby obviously competes with other interventions in, for example, HIV/AIDS, maternal and child health for scarce health care resources. These and related issues were discussed in a workshop on the future plans of scaling up PAL in Nepal, and the present study was conducted alongside this workshop. It thus provides a unique opportunity to compare a rational approach to decision making with those discussions that are actually taking place.
| Methods |
|---|
|
|
|---|
Discrete choice experiment
In a DCE, respondents choose their preferred option from sets of hypothetical options, each consisting of a bundle of attributes or criteria that describe the option in question, with each criterion varying over a range of levels. The criteria are constant in each scenario, but the levels that describe each criterion may vary across options. Analysis of the options chosen by respondents in each set reveals the extent to which each criterion is important to the decision at hand (Ryan and Gerard 2003
; Kjaer 2005
). Running a DCE involves a number of steps, and these are discussed in turn.
Definition of criteria and levels
We organized two group discussions to identify the relevant criteria and related levels to be included in the DCE, which included a total of seven policy makers and people otherwise involved in regional health care programmes. A wide range of criteria were mentioned, and summarized in a number of categories. Some criteria put forward in these discussions were related to common aspects of all interventions, such as the need to improve access to health care. In as far as these criteria related equally to all interventions, they were not retained in the subsequent research. Furthermore, the economic impact of a disease was discussed as a priority criterion and was later summarized into including a specific age category for the income-generating population (considered as those between 15 and 59 years). The group discussion resulted in the identification of six criteria with associated levels. These are shown in Table 1 (together with their coding for the regression analysis).
|
Experimental design and data collection
On the basis of four criteria measured at two levels, and two criteria at three levels, 144 unique scenarios can be defined for inclusion in a full factorial experimental design in DCE (Ryan and Gerard 2003
; Kjaer 2005
). However, to avoid informational overload, the use of a fractional factorial design including a limited number of scenarios is often suggested (Ryan and Gerard 2003
). Our fractional factorial design included a subset of 18 scenarios (representing an orthogonal array), to allow for estimation of all main effects (Sloane 2005
). Each of these 18 scenarios was paired to its mirror image so as to retrieve the maximum information from each choice. An example of a pair of scenarios is given in Figure 1. The DCE survey was administered during three sessions with policy makers in health and health professionals involved in mid-level health care management and public health provision. All respondents had at least 2 years professional experience in the health sector. The sessions were organized around a workshop on the future plans of scaling up PAL in Nepal. Respondents were familiarized with the conceptual framework and worked through a number of examples before they embarked on the DCE exercise. In total, 66 respondents (43 men and 23 women, mean age 38.6 years) made choices between 18 sets of scenarios.
|
Data analysis
All levels for all criteria were qualitative and were dummy coded. Dummy coding involves that a criterion with L qualitative levels is transformed into L-1 dummy variables in which each dummy is set equal to 1 when the qualitative level is present and set equal to 0 if it is not. Binary logistic regression models were used to analyse the response data, with the following equation being estimated:
|
| (1) |
the unobservable error term and all other variables are as defined in Table 1. To control for differences in attractiveness of scenarios we added dummies for scenarios to the equation. We assume a main-effects additive probability model, which derives a linear combination of the weights of each level of all criteria. It is standard practice in a DCE to assume such a linear function. It has been observed that, given the inclusion of dummy variables, this model does not impose an interval scale or ordinality on the relationship between the criteria and predicted probability scores (Kjaer 2005The results are presented as regression coefficients, average marginal effects and relative contributions. Regression coefficients indicate the sign of the effect of a variable on the probability of selection of an intervention, but have no direct quantitative interpretation here. Average marginal effects can be quantitatively interpreted and reflect the change in probability of selection of an intervention following a change in a single variable. The average marginal effects are computed by taking the difference in estimated probability of P with and without the variable, while holding the distribution of the other variables at their sample value, and then taking the sample mean of these differences. The relative contributions indicate the contribution of one criterion to the variation in preferences explained by the model (Efron's R2). These contributions are calculated by computing Efron's R2 with the model that excludes the criterion of interest while holding the coefficients of the other criteria constant. This procedure allows us to evaluate the contribution of criteria irrespective of the number of levels they have.
Composite league table
Next, we considered a large set of 33 interventions as comparators for the PAL programme. These interventions are related to child and maternal health, TB and HIV/AIDS, and address an important part of the burden of disease in Nepal. They cover the disease areas as put forward in the health-related Millennium Development Goals adopted by the United Nations including Nepal (United Nations 2000
), and we therefore regard them as relative priorities in health in Nepal. The cost-effectiveness of interventions was based on work by the WHO-CHOICE project (Evans et al. 2005a
). Information on poverty reduction was retrieved from the World Health Report 2002: Reducing risks, promoting healthy life (WHO 2002
), whereas information on severity of disease, number of potential beneficiaries and individual health benefits was based on consultation of disease models employed in WHO-CHOICE. In addition, the information on PAL was based on the results from an economic evaluation (Shrestha et al. 2005
), and associated population-level disease model (Samir et al. 2005
), whereas information on poverty reduction was retrieved from the World Health Report 2002 (WHO 2002
) (the list of interventions and whether a qualitative level of a variable is present is shown in Table 3, where 0 denotes the absence and 1 the presence of a level).
|
Next, the probability of selection was estimated for each intervention using equation (1). Subsequently, all interventions were rank ordered on the basis of this probability of selection, on the assumption that it relates in a positive way to the attractiveness of that intervention. The results can then be interpreted as a composite league table, with the most attractive interventions at the top and the least attractive interventions at the bottom.
| Results |
|---|
|
|
|---|
The results of the discrete choice experiments are shown in Table 2. All coefficients were significant and their signs had the expected direction. The marginal effects show, for example, that interventions that target severe diseases have a 19.7% higher probability of being selected than interventions that target non-severe diseases, other things being equal. Also, interventions that target people of the middle-age group have a 12.7% higher probability of being selected than interventions that target the young, but those that target the elderly have a 30.5% lower probability of being selected. Overall, interventions that target severe diseases, many beneficiaries, people of the middle-age group, have large individual health benefits, lead to poverty reduction, and are very cost-effective have a higher probability of being selected than interventions without (one of) those characteristics.
|
The relative contributions show that age of target group is the most important criterion, followed by individual health benefits, severity of disease, cost-effectiveness and number of potential beneficiaries. The least important criterion is poverty reduction. The model explained 31% of all observed variance in preference.
The composite league table shows that interventions with the highest probability of selection are in TB control, followed by oral rehydration therapy for diarrhoea and case management of pneumonia in child health, and several interventions in HIV/AIDS control including the provision of antiretroviral therapy (Table 3). PAL ranks 13th. On the basis of cost-effectiveness information alone, the rank ordering would be less differentiated, and PAL would be ranked 27th only (not in Table).
| Discussion |
|---|
|
|
|---|
This paper has shown the feasibility of simultaneously accounting for efficiency, equity and societal concerns in prioritization decisions, and its potentially large impact on priority setting. For example, whilst PAL would be given low priority under pure efficiency considerations, it would be ranked much higher if the policy maker was also concerned about severity of disease, number of potential beneficiaries, age, individual health benefits, and poverty reduction. By showing the relative importance of the different criteria, policy makers can clearly see the implications of tradeoffs between different concerns on prioritization decisions.
The results seem to follow common perceptions on people's preferences. For example, respondents give priority to (interventions for) people of middle age compared with the young and old, and this is in line with the concept of age-weighting in the calculation of disability-adjusted life years (DALYs) in burden of disease studies (Murray and Lopez 1997
).
The evaluation of PAL in the period 20022004 was led by the National Tuberculosis Centre (NTC), and was seen as one of its potential priority programmes to combat the increase of TB (ten Asbroek et al. 2005
). An important operational issue in the implementation is the budget impact of PAL incurred by its training costs (Samir et al. 2005
). Overall, PAL leads to significant cost savings but these fall on the patient and not on the budget of the government. If no additional resources become available, it is not sure whether PAL will be scaled up to the country as a whole. This situation seems to contradict with our findings that indicate PAL as a priority programme.
One political explanation for this apparent contradiction is the rigidity of the resource (re)allocation process (Murray et al. 2000
). If PAL was indeed a priority in Nepal, vested interests and the influence of political groups could make the reallocation of resources from unattractive programmes to more attractive, priority, programmes, such as PAL, difficult. This would reiterate the importance of rational priority setting processes as proposed in this paper. Another, merely methodological, explanation is the absence of high-level decision-makers in the working groups, to identify criteria for priority setting. As a consequence, the views of those who take the final programme decisions were not included to the optimal extent, and led, for example, to the omission of criteria like budget-impact in the analysis. PAL might not have been a priority programme in our findings if budget-impact had been included as a criterion.
The composite league table should be considered as illustrative only. First, the interventions that were included as comparators for the PAL programme do not reflect the variety and nature of all interventions that are currently implemented in Nepal. For example, many of the child interventions are presented here in isolation, but are carried out in Nepal in the Integrated Management of Childhood Illnesses (IMCI) programme. Secondly, not all information on criteria and levels was available for all interventions to construct the composite league table. This also applies to the PAL programme: the economic evaluation only included the short-term gains in quality of life, not the long-term gains in mortality reduction (Samir et al. 2005
), and this may have underestimated its health effects and cost-effectiveness, and consequently its ranking in the composite league table. On the other hand, if we had classified lung health diseases as non-severe, PAL would be rank ordered 26th in the present list.
This study has elicited the health preferences of policy makers in Nepal, on the assumption that these legitimately represent the preferences of the nation as a whole. Whether preferences of members of the general public are more apt in this context is a topic of much debate (Wiseman 2005
). To foster that discussion, future studies could compare preferences of policy makers with those of members of the general public.
This paper has introduced the concept of a composite league table to guide priority setting in health. It thereby responds to the recent call for the development of MCDA for priority setting (Mills et al. 2004
). This can be an important step forward to a rational approach to priority setting in developing countries. The approach meets the conditions for successful MCDA in health as recently put forward by Baltussen and Niessen (2006
): in a quantitative manner, a comprehensive set of criteria (as identified though group discussions) is considered simultaneously allowing trade-offs (in the DCE) and rank ordered (on the basis of their probability of selection). However, the priority-setting process was not embedded in the organizational context (e.g. Ministry of Health in Nepal) and its results have not been discussed with a range of stakeholders as organized, for example, in an advisory panel. This has indeed limited the relevance of results for actual policy making, and follow-up research should aim to embed the approach in that context.
The overall methodological approach is generalizable to other settings. Its application to another country would require the identification of priority-setting criteria relevant to that country, including marginal effects to derive probabilities of selection for the interventions, to arrive at a country-specific composite league table.
| Conclusion |
|---|
|
|
|---|
This explorative analysis suggests that the lung health programme is among the priorities in Nepal when taking into account a range of relevant criteria for priority setting. The multi-criteria approach can be an important step forward to rational priority setting in developing countries.
| Acknowledgements |
|---|
We thank the participants in the DCE for their kind cooperation. We are grateful to Dr K B Shrestha, Dr P Malla and Dr Gunneberg of the National Tuberculosis Centre of Nepal, and Prof. R P Gartoulla of the Nepal Institute of Health Sciences for facilitating the DCE presented in this study.
| References |
|---|
|
|
|---|
Baltussen R and Niessen L. 2006. Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Effectiveness and Resource Allocation 4:14.[CrossRef]
Baltussen R, Stolk E, Chisholm D, Aikins M. 2006. Towards a multi-criteria approach for priority setting: an application to Ghana. Health Economics 15:68996.[CrossRef][Web of Science][Medline]
Evans DB, Lim SS, Adam T, Tan-Torres Edejer T. and the WHO-CHOICE MDG Team. 2005a. Evaluation of current strategies and future priorities for improving health in developing countries. British Medical Journal 331:145761.
Evans DB, Lim SS, Adam T, Tan-Torres Edejer T. and the WHO-CHOICE MDG Team. 2005b. Methods to assess the costs and health effects of interventions for improving health in developing countries. British Medical Journal 331:113740.
Global Fund. 2005. The Global Fund to Fight AIDS, Tuberculosis and Malaria. Online at: [http://www.theglobalfund.org/en/], accessed 24 June 2005.
Ham C. 1997. Priority setting in health care: learning from international experience. Health Policy 42:4966.[CrossRef][Web of Science][Medline]
Hirschberg S, Dones R, Heck T, et al. 2004. Sustainability of electricity supply technologies under German conditions: a comparative evaluation. PSI Report No. 0415.Villigen, Switzerland Paul Scherrer Institut.
James C, Carrin G, Savedoff W, Hanvoravongchai P. 2005. Clarifying efficiency-equity tradeoffs through explicit criteria, with a focus on developing countries. Health Care Analysis 13:3351.[CrossRef][Web of Science][Medline]
Kapiriri L and Norheim OF. 2004. Criteria for priority setting in health care in Uganda: exploration of stakeholders values. Bulletin of the World Health Organization 82:1729.[Web of Science][Medline]
Kapiriri L, Norheim OF, Heggenhougen K. 2003. Using burden of disease information for health planning in developing countries: the experience from Uganda. Social Science and Medicine 56:243341.[CrossRef][Medline]
Kapiriri L, Arnesen T, Norheim OF. 2004. Is cost-effectiveness analysis preferred to severity of disease as the main guiding principle in priority setting in resource poor settings? The case of Uganda. Cost Efficiency and Resource Allocation 2:1.[CrossRef]
Kjaer T. 2005. A review of the discrete choice experiment with emphasis on its application in health care. Health Economics Papers 2005: 1.Odense, Denmark University of Southern Denmark.
Land and Plant Nutrition Management Service (LPNMS). 2004. MCDA Multi-Criteria Decision Analysis techniques, using the Aspiration-Led Decision Support (ALDS) approach. Online at: [http://www.fao.org/ag/agl/agll/infotech.htm], accessed 22 December 2005.
Linkov I, Varghese A, Jamil S, et al. 2004. Multi-criteria decision analysis: a framework for structuring remedial decisions at contaminated sites. In Linkov I and Ramadan A (Eds.). Comparative risk assessment and environmental decision making. Amsterdam: Kluwer, pp. 1554.
Martin DK and Singer PA. 2000. Priority setting and health technology assessment: beyond evidence based medicine and cost-effectiveness analysis. In Ham C and Coulter A (Eds.). The global challenge of health care rationing.Buckingham, UK Open University Press.
McDaniel C and Gates R. 2005. Marketing research essentials. 5th Hoboken, NJ John Wiley & Sons Inc.
McIntyre D and Gilson L. 2000. Redressing disadvantage: promoting vertical equity within South Africa. Health Care Analysis 8:23558.[CrossRef][Web of Science][Medline]
Mills A, Bennett S, Bloom G, González-Block MA, Pathmanathan I. Alliance for Health Policy and Systems Research. 2004. Strengthening health systems: the role and promise of policy and systems research.Geneva Alliance for Health Policy and Systems Research.
Murray CJ and Lopez AD. 1997. Alternative projections of mortality and disability by cause 19902020: Global Burden of Disease Study. The Lancet 349:1498504.
Murray CJ, Evans DB, Acharya A, Baltussen RM. 2000. Development of WHO guidelines on generalized cost-effectiveness analysis. Health Economics 9:23551.[CrossRef][Web of Science][Medline]
National Economic Research Associates. 2000. Multi-criteria analysis manual. Online at: [http://www.communities.gov.uk/index.asp?id=1142253], accessed 27 February 2007.
Nelson EA, Olukoya A, Scherpbier RW. 2004. Towards an integrated approach to lung health in adolescents in developing countries. Annals of Tropical Paediatrics 24:11731.[CrossRef][Web of Science][Medline]
Ryan M and Gerard K. 2003. Using discrete choice experiments to value health care programmes: current practice and future research reflections. Applied Health Economics and Health Policy 2:1.[Medline]
Samir KC, Baltussen R, Shrestha N, Niessen L. 2005. Cost-effectiveness of PAL in Nepal, a modelling approach. iMTA working paper 6.Rotterdam, Netherlands Institute of Medical Technology Assessment.
Scherpbier RW, Hanson C, Raviglione M. 1998. Adult Lung Health Initiative: Basis for the development of algorithms for assessment, classification and treatment of respiratory illness in school-age children, youths and adults in developing countries. Document WHO/TB/98.257.Geneva World Health Organization.
Segal L and Chen Y. 2001. Priority setting models for health. CHPE Working paper 119.Clayton, Australia Centre for Health Program Evaluation, Health Economics Unit, Monash University.
Shrestha N, Baltussen R, Samir KC, Niessen L. 2005. Costs, health effects and cost-effectiveness of respiratory guidelines in Nepal: a randomized controlled clinical trail. iMTA working paper 8.Rotterdam, Netherlands Institute of Medical Technology Assessment.
Sloane NJA. 2005. A library of orthogonal arrays. Online at: [http://www.research.att.com/~njas/oadir/], accessed 28 April 2005.
Stolk EA, van Donselaar G, Brouwer WB, Busschbach JJ. 2004. Reconciliation of economic concerns and health policy: illustration of an equity adjustment procedure using proportional shortfall. Pharmacoeconomics 22:1097107.[CrossRef][Web of Science][Medline]
ten Asbroek AH, Delnoij DM, Niessen LW, et al. 2005. Implementing global knowledge in local practice: a WHO lung health initiative in Nepal. Health Policy and Planning 20:290301.
Tsuchiya A. 1999. Age-related preferences and age weighting health benefits. Social Science and Medicine 48:26776.[CrossRef]
United Nations. 2000. United Nations Millennium Declaration General Assembly resolution A/RES/55/2. New York United Nations 18 September 2000.
WHO. 2002. World Health Report 2002: Reducing risks, promoting healthy lifeGeneva World Health Organization.
WHO. 2004. Practical approach to lung health. Geneva World Health Organization Online at: [http://www.who.int/tb/dots/pal/en/].
WHO Commission on Macroeconomics and Health. 2001. Macroeconomics and health: investing in health for economic development. Report of the Commission on Macroeconomics and Health.Geneva World Health Organization.
Wiseman V. 2005. Comparing the preferences of health professionals and members of the public for setting health care priorities: experiences from Australia. Applied Health Economics and Health Policy 4:12937.[CrossRef][Medline]
In Wikler D (Ed.). Fairness and goodness in health.2004. Geneva World Health Organization.
Accepted for publication 20 February 2007.
![]()
CiteULike
Connotea
Del.icio.us What's this?
This article has been cited by other articles:
![]() |
L. J Mangham, K. Hanson, and B. McPake How to do (or not to do) ... Designing a discrete choice experiment for application in a low-income country Health Policy Plan., March 1, 2009; 24(2): 151 - 158. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||


