Health Policy and Planning Advance Access published online on August 13, 2008
Health Policy and Planning, doi:10.1093/heapol/czn028
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Toward a grounded theory of why some immunization programmes in sub-Saharan Africa are more successful than others: a descriptive and exploratory assessment in six countries
1Global Immunization Division, National Center of Immunization, Centers for Disease Control and Prevention, Atlanta, Georgia, USA.
2Health, Nutrition and Population, Human Development Network, The World Bank, Washington DC, USA.
3Booster Program for Malaria Control, The World Bank, Washington, DC, USA.
4Human Development Department, Africa Region, The World Bank, Washington, DC, USA.
5Human Development Department, South Asia Region, The World Bank, Washington, DC, USA.
*Corresponding author. Health, Nutrition and Population, Human Development Network, The World Bank, 1818 H Street NW, MSN G8-801, Washington, DC, 20433. Tel: +202–458–0138. Fax: +202–522–3235. E-mail: jnaimoli{at}worldbank.org
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionConclusion/future directionsEndnotesReferences |
|---|
The question of why some immunization programmes in sub-Saharan Africa are more successful than others is an intriguing one, but not one that is frequently raised or investigated. Borrowing techniques from both performance benchmarking and positive deviance inquiry, we explored this question in six countries. We first set out to define for a systematic sample of countries the key constructs commonly associated with improving immunization coverage, using an inductive, ‘insider’ point of view. We then explored their utility in generating hypotheses about coverage differences across countries through a preliminary application of the measures of these constructs to the countries in this sample. Our findings suggest that there are different paths to success, and that not only what countries do, but how they execute their programmes, seem to make a difference in coverage outcomes. In some cases, extramural, contextual factors may also help to explain these differences. We discuss several hypotheses generated by our study, identify methodological limitations, and recommend improvements to the methods we used. Similar formative studies are needed to validate our preliminary hypotheses, to generate new ones, and to raise our level of confidence in the early policy implications that we see emerging from our preliminary work in this area. Eventually, testing of the hypotheses generated by this and other formative studies could generate a robust theory of why some programmes are more successful than others, a phenomenon likely to be relevant to other child and maternal health programmes in sub-Saharan Africa.
Key Words: Childhood immunization, sub-Saharan Africa, policy implementation, programme implementation, grounded theory
KEY MESSAGES
|
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusion/future directionsEndnotesReferences |
|---|
The current apprehension within the global health community that available, affordable and potentially life-saving technologies are not reaching those who need them most is not a new concern (Wagstaff and Claeson 2004
; Bryce and Victora 2005). This has been a disquieting refrain since the advent of the so-called ‘Child Survival Revolution’ of the late 1970s, which was driven, largely, by the ‘twin-engine’ technologies of vaccines and oral rehydration salts (Claeson and Waldman 2000). What is new, however, is a broadly shared responsibility for doing something about the problem, now and in the future, as reflected in the commitment by all countries and the leading development institutions to achieving the Millennium Development Goals (MDGs) by 2015. MDG #4 aims to reduce child mortality by half from 1990 levels; unfortunately, the consensus appears to be one of inadequate progress toward reaching this goal, particularly where improvement is needed most, namely in sub-Saharan Africa (UN 2005; Bryce et al. 2006).
Increasingly recognizing that a central challenge in reaching this goal is to sustain sufficient coverage of the world's poor with an ever-expanding number of proven disease prevention and health promotion interventions (Bryce and Victora 2005), the global health community is searching for ‘best practices’ in achieving health results at scale. A logical place to begin is with childhood immunization, which has a long and notable track record in sub-Saharan Africa, and which is at the forefront of innovation and new technology introduction (Martin and Marshall 2003; Muraskin 2004). Curiously, the normative advice usually given to countries about improving routine coverage tends to be informed as much by ideology and conventional wisdom as it is by systematic evidence of what works in different settings under different conditions (Gauri and Khaleghian 2002; Pegurri et al. 2005). Practical theory building about why some programmes are more successful than others that is grounded in how these programmes are actually executed, can verify, inform and enrich this guidance, and offer new insights. We borrow techniques from both performance benchmarking (Camp and Tweet 1994) and positive deviance inquiry (Marsh et al. 2004)—both of which try to explain why certain organizations, programmes, communities or individuals experience better outcomes than their peers under similar circumstances—to understand better what appears to work best in immunization.
The purpose of our study was both descriptive and exploratory. We first set out to define for a systematic sample of sub-Saharan countries the key constructs commonly associated with improving immunization coverage in developing and developed countries, using an inductive, ‘insider’ point of view. We then explored their utility in generating hypotheses about coverage differences across multiple settings through a preliminary application of the measures of these constructs to the countries in this sample. We present and discuss several hypotheses generated by the research, identify methodological limitations and recommend improvements to our methods. Finally, we identify additional formative and confirmatory research required to raise our level of confidence in the early policy implications we see emerging from our preliminary work in this area, and to build a robust, grounded theory of why some programmes are more successful than others.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusion/future directionsEndnotesReferences |
|---|
Identifying and defining core programme implementation constructs
Preliminary desk review and preparation for fieldwork
To identify the key constructs commonly associated with improving coverage, we consulted two review articles on programmatic interventions to improve or expand childhood immunization coverage in developed and developing countries (Briss et al. 2002; Pegurri et al. 2005). From this review, we adopted four core functions to guide our definitional and data collection activities: supply, demand, financing and the governance/institutional framework. We used a multi-phased, purposive sampling strategy to select countries in which to operationalize these functions. Selection criteria included immunization coverage history, population size, status of World Bank support to the health sector, and feasibility of data collection.
We chose coverage of children under 5 years of age with three doses of the diphtheria-pertussis-tetanus vaccine (DPT3) as our outcome of interest. DPT3 is not only a barometer of a country's capacity to execute successfully its routine immunization programme, but also a proxy for the strength of its health service delivery system (Bos and Batson 2000). Our data source was the WHO/UNICEF Estimates (WHO/UNICEF 2004). These Estimates are based on all available coverage information reported by every country, including standard administrative data, recent household coverage survey data and official country estimates of coverage.1 Survey data are weighted more heavily than reported data in the estimation process, and each year a WHO/UNICEF team updates previous years’ estimates in light of the most recent reporting. At the beginning of our study (2003), Estimates for most sub-Saharan countries were available through 2002; consequently, we chose the period 1997–2002 as a reasonable representation of recent coverage history, and we examined measures of central tendency (mean coverage) and change (linear coverage trend) during this period.2 Complete data were available for 43 of the 47 countries (data were not available for Liberia, Sierra Leone, Sao Tome and Principe, and Cape Verde).
We created a scatter plot of mean coverage by the linear coverage trend for the 6-year period (Figure 1). We divided the plot into four quadrants by drawing two lines—each of which was anchored at the median value of the x axis (average annual percentage point change of 0.3) and the y axis (58% average coverage)—through the data points. Substantial variation in coverage was apparent. We then assigned to each of the four patterns of coverage history a performance classification: high, medium-high, medium-low and low (Table 1).
|
|
Within each classification, countries eligible for selection were those with medium or large populations (over 2 million), for which the findings of the study would likely contribute to enhancing World Bank-supported activities in the health sector, and which expressed interest and willingness to participate. We selected from each of the four classifications one country that satisfied all the selection criteria. Two additional countries that met all these criteria, but also exhibited an exceptional characteristic, were also selected. Six was the maximum number of countries that we could investigate with the resources at our disposal. The final sample comprised the following countries: Ghana and Rwanda (high performance); Malawi (medium-high performance); Ethiopia and Mauritania (medium-low performance); and Cameroon (low performance). Rwanda (recently emerging from conflict and genocide) and Mauritania (unusually dramatic change in its linear coverage trend) were selected on the basis of exceptionality.
Fieldwork
Two field investigators collected data in the six countries in accordance with a standard protocol. Neither worked in the World Bank's Africa Region or had management responsibility for any Bank investment operation in any of these countries. Data collection in each country included open-ended discussions with key informants, individually and in group, and a comprehensive review of all relevant documentation from the study period.
Key informants included all individuals who had intimate knowledge of implementation during this period as judged by the current national immunization programme manager. These individuals included Ministry of Health officials, as well as World Health Organization (WHO) and UNICEF staff, some of whom were previously employed by the government and directly or indirectly involved with the immunization programme. There were no major problems of access to key informants. After reviewing recent immunization coverage history with informants, investigators asked them to explain the programme's performance, using the four programme functions as guides, and to mention anything else that they believed might have had a bearing on performance. Investigators recorded all informants’ descriptions and explanations in their field journals. The discussions were far-ranging and carried out in a spirit of collegiality and mutual learning.
The historical documentation for each country consisted of official government policies, plans and reports; donor-funded reports; and all collaborative assessments, evaluations, surveys (such as any recent Demographic and Health Surveys) or audits (such as Data Quality Audits). As with the informant interviews, the four programme functions and any additional contextual factors that may have influenced coverage guided the documentary review. There were no major access or retrieval problems. Investigators made extensive annotations to the documentation. In most instances, we found a high level of convergence between what was reported by informants and what had been written.
One investigator collected data in four countries; the other, with an assistant, worked in two. The duration of fieldwork in each site was approximately 5–7 days. Only the Ghana review was conducted off-site—interviews were conducted by telephone and the historical documentation was assembled and reviewed at World Bank offices in Washington, DC.
Refinement of constructs and final adoption of implementation measures
During a two-week period, the investigators developed intermediary field records for all countries based on the information in their field journals and their annotations to programme documentation. The investigators then composed succinct draft case studies by looking for patterns and themes in their records. Approximately 4–6 weeks were required to construct a case narrative. At least one key informant from each country reviewed the draft, and all drafts were reviewed by the immunization focal point for the World Bank's Africa Region. With few exceptions, the reviewers found the case studies to be accurate and clear, and they suggested minimal modifications.
After reading all the cases, we more precisely defined the supply function as two distinct functions: ‘management’ and ‘service delivery strategy’. We then derived a standard set of measures for each function, based on a content analysis of our benchmark countries—the high performers. We reviewed the four remaining cases to ensure consistency and the presence of a minimum amount of information for each measure. In the event of missing or insufficient information, we revisited the original source material and further revised the case to ensure better consistency across measures and countries. A final list of locally defined measures for each function was adopted (Box 1).
| Box 1 Definitions of programme functions Governance/Institutional Framework
Management
Strategy
Demand
Financing
|
Analysis
After finalizing the list of measures for each function, we developed a standard scoring sheet to aid in determining the extent to which each measure was present in each case in our sample. The four-point ordinal scale we adopted was as follows: ‘measure strongly present’ (value = 4); ‘measure partly present’ (value = 3); ‘measure weakly present’ (value = 2); and ‘measure absent’ (value = 1). A panel of six analysts, including the two field investigators and four World Bank staff members (all with experience and expertise in immunization or health service delivery in Africa), independently evaluated the cases. After reading each case, each analyst, with the aid of the scoring sheet, assigned a numerical value to his or her judgment of the degree to which each of the measures for each function was in evidence in each country. We aggregated the scoring of all analysts. In the event of large discrepancies on individual measures, which were rare, we convened the panel to discuss these differences and to reach consensus on a reasonable compromise. Analysts were not blinded to the performance categories of their cases.
Once substantial agreement on the scoring had been achieved, we calculated ‘measure point averages’ (i.e. the average of the five raters’ scores for each measure) and ‘function point averages’ (i.e. by adding all the scores of the measures of each function and dividing by the total number of measures of the function). Each function and within-function measure was equally weighted. For the financing and demand functions, we did not enter certain scored measures into the overall function point average due to lack of agreement among investigators. We then entered these averages into a ‘case-ordered function-outcome matrix’. To facilitate recognition of patterns across cases we converted the numerical scores into ‘letter-grade’ equivalencies (ranging from A to D) patterned after the ‘grade point average’ system commonly used in institutions of higher learning in the US (Table 2).
|
We arbitrarily defined as ‘reasonably good’ any function or measure with a letter grade of B or better. We defined as ‘deficient’ any function or measure with a letter grade less than B. The decision rule for ‘programmatically important’ differences between functions and measures was a two-level difference in letter grade (i.e. the difference between an A and B+, an A– and a B, a B+ and a B–, etc.). We shared and discussed our results with representatives of all participating countries during a series of videoconferences.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusion/future directionsEndnotesReferences |
|---|
Programme factors
The results of the review panel's evaluation of the case studies are presented in Table 3. A graphic summary of each country's functional performance is presented in Figure 2. The summary that follows draws upon Table 3, Figure 2 and, in some instances, selected case study text. The complete set of cases is available upon request.
|
|
High performers: Rwanda and Ghana
Rwanda and Ghana performed reasonably well on all five programme functions. There were, however, some programmatically important differences between them on certain within-function measures. For example, Rwanda fared better than its peer on the capacity for action and coordination measures of governance, as well as the human resource measure of management. While the fixed service delivery strategy was dominant in Rwanda, outreach was dominant in Ghana. Ghana's dependence on external financing was considerable; Rwanda's less so. Ghana's programme benefited from the successful execution of a Sector-Wide Approach (SWAp) to development partner support, which was implemented with both pooled and earmarked funds. There was no SWAp mechanism in Rwanda. Finally, they differed with respect to sustaining initial demand for services and the degree of effort for demand-creation activities (intense in Rwanda, weak in Ghana).
Medium-high performer: Malawi
Malawi performed reasonably well on two functions: governance/institutional framework and financing. On certain measures of financing, Malawi was programmatically similar to the high performers: the degree of dependence on external financing was comparable with Ghana, and the absence of a SWAp comparable with Rwanda. Where Malawi diverged from them was in its performance on management, service delivery strategy and demand. Malawi did not perform reasonably well on any of these functions.
Regarding management, the key differences between Malawi and the high performers appeared to be in cold chain, vaccine, injection supply and information management. Human resources management, however, was as problematical for Malawi as it was for Ghana, and all three countries suffered from comparable transport management difficulties. On strategy, Malawi was comparable with Rwanda on outreach, but differed from both Rwanda and Ghana on service delivery from fixed sites. On demand, Malawi was comparable with both high performers with respect to initial seeking of services; however, it differed from both with respect to sustained demand. Malawi's elevated dropout occurred despite more intense efforts than Ghana to encourage and promote the population's use of immunization services, a strategy also employed by Rwanda, but with greater intensity.
Medium-low performers: Ethiopia, Mauritania
Ethiopia: Ethiopia did not score reasonably well on any of the five functions. The differences between Ethiopia and both high performers, and Ethiopia and Malawi, were programmatically important on all functions. These differences, however, mask similarities on certain measures across all functions. For example, Ethiopia is comparable with Malawi on measures of policy alignment (governance/institutional framework), vaccine management (management), vaccination from fixed sites (strategy), external financing and the mechanism for coordinating external resources (financing), and sustained demand (demand). Ethiopia is even comparable with one or both of the high performers on certain measures, such as policy alignment, external financing and demand-creation activities. For example, Ethiopia's dependency on external financing and its execution of the programme within a SWAp or SWAp-like environment were comparable with the high-performing Ghana and the medium-high performing Malawi.
Mauritania: Although Mauritania is paired with Ethiopia in this classification, it had a very different implementation experience. Mauritania's coverage history, including its 6-year average coverage (42%) and linear trend (average annual percentage point change of 12.1), as well as its ratings on the five functions, was heavily influenced by a bi-modal implementation experience. Between 1997 and 1999, Mauritania's mean coverage was approximately 23% and all evidence points to a programme commonly characterized by government officials and partners as ‘moribund’. During the next 3 years, spurred by what informants described as a ‘re-launch’ of the programme in 2000, mean coverage reached 61%, and there was substantial evidence that the programme functioned in a manner quite different from the previous 3 years. Consequently, Mauritania's average implementation experience for the period presents a picture that quite closely mirrors that of Malawi: reasonably good governance on a level with the high performers; reasonably good financing, albeit different from the high performers in a programmatically important way; deficiencies in management, strategy and demand, with important differences with the high performers; yet, certain similarities with the high performers, particularly with respect to measures of financing (external dependency and donor coordination), management (transport constraints, in particular), outreach services (similar to Rwanda), and demand (similarities with Ghana on demand-creation activities).
Low performer: Cameroon
Cameroon, like Ethiopia, did not score reasonably well on any of the five functions. Cameroon's differences with the high performers were programmatically important on all functions. When compared with the three medium performers, Cameroon appears to be most unlike Malawi (programmatically important differences on four of the five functions), somewhat more similar to Mauritania (differences on two of the five functions), and most like Ethiopia (difference on one function only—financing). Again, there is heterogeneity in performance on measures across multiple functions. For example, Cameroon's dependency on external financing is similar to the high-performing Rwanda and the medium-performing Mauritania. The execution of its programme within a SWAp or SWAp-like environment is similar to the high-performing Rwanda. The low level of intensity for demand-creation activities in Cameroon was comparable with that of the high-performing Ghana and the medium-performing Mauritania and Ethiopia.
Contextual factors
Key informants in each country reported a series of extramural, contextual factors (i.e. outside the control of the programme) that they believed to have contributed to immunization performance during the study period. Health officials in Mauritania attributed progress in implementation between 2000 and 2002 to immunization's movement to the centre of macro-economic discussions about poverty reduction and debt relief (Box 2). Increased demand for immunization services in Rwanda was reported to be a result, in part, of security measures put in place in the post-genocide/post-conflict period. A major drought, increasing brain drain and a worsening HIV/AIDS epidemic in Malawi were reported to have exacerbated a human resource crisis in the health sector, which was reported to have impacted directly on the management component of the immunization programme. Ethiopia's peace treaty with Eritrea reportedly resulted, ultimately, in benefits for immunization. Cameroon's financial crisis of the 1980s and 1990s left a legacy of severe under-funding and under-performance in the health sector, which has only recently shown signs of rebounding.
| Box 2 The Mauritanian experience The implications of immunization's movement to the centre of macro-economic discussions in Mauritania included improved programme governance and a stronger institutional framework resulting from an official ‘re-launch’ of immunization, and increased domestic financing, chiefly through resources from the Heavily Indebted Poor Countries Initiative (HIPC). Although deficiencies in the service delivery strategy were noted, increased emphasis upon outreach, primarily mobile and door-to-door services in rural areas, and multi-antigen vaccination days in urban areas, were credited with contributing to Mauritania's substantial progress in coverage. The case study emphasizes the important role of political pressure from the highest levels in improving DPT3 coverage, at all costs, and the fact that the HIPC target was reached in the absence of a major overhaul of the existing programme. Today, in the aftermath of the fervour that accompanied the HIPC-driven pursuit to reach 70% DPT3 coverage by 2002, some public health officials are concerned that these gains may be temporary and are fearful of the potentially negative, albeit unintended, consequences of the activities used to achieve them (i.e. possibly undermining future efforts to increase the population's use of services at health facilities, and to maintain health worker motivation in delivering routine services in the absence of a political imperative).
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusion/future directionsEndnotesReferences |
|---|
The establishment in 2000 of GAVI, a global partnership for immunization, and the considerable attention to polio eradication, measles mortality reduction, new vaccines, and innovative financing for immunization in both the popular press and the scientific literature (Hardon and Blume 2005
; Otten et al. 2005; Phillips 2006) might lead one to conclude that the immunization agenda in Africa is being satisfactorily addressed. On the contrary, early in our study we discovered considerable variation in immunization coverage. A different DPT3 performance rating scheme, using more recent Estimates data from a larger sample of sub-Saharan countries for the period 2000–04, confirms this finding (Table 4). Even in the case of a well-established intervention such as immunization, there are serious coverage shortfalls in the region yet ample room for improvement.
|
Although our definition of and findings about implementation are not transferable or generalizable to all of sub-Saharan Africa, what we learned from these six countries, albeit preliminary and suggestive, may nonetheless be instructive and relevant to others in the region. First, what distinguished the two high performers from all other countries in our exercise was a robust implementation of their immunization programmes. Only Ghana and Rwanda combined reasonably good governance and a solid institutional framework with reasonably good management, service delivery, financing and demand. Although we did not study directly how these countries allocated resources across programme functions, this question merits further investigation. Differences between these two countries in various within-function measures suggest there may be different paths to success. Considering the constrained resource environments of most African countries, the ever-changing landscape in development assistance, and competing public health priorities, it remains to be seen whether these countries can sustain robust implementation.
One way that multilateral and bilateral partners in immunization may help is by offering countries performance-based incentives without earmarking their assistance for specific functions. GAVI's Immunization Services Support (ISS) fund is an example of such support. A recent assessment indicated that recipient countries appreciated the flexibility these funds offered, and they disbursed them in accordance with locally identified needs (Chee et al. 2004). Progress toward robust implementation may be impeded, however, when partners earmark their support for specific diseases or selected functions of the immunization programme. Opportunity costs to the programme for obtaining and using earmarked assistance may be high and the sustainability of the activities this assistance supports uncertain.
Second, reasonably good performance on at least some functions (Malawi and Mauritania) appeared to be more beneficial to coverage than deficient performance across all (Ethiopia and Cameroon3); doing a few things well, however, did not appear to be sufficient to ensure a high level of sustained success. In the cases of Malawi and Mauritania, it is interesting to note that the same two functions—governance/institutional framework and financing—interact in a positive way to either protect the programme from further slippage (Malawi) or to stimulate progress in the absence of reasonably good performance on other functions (Mauritania). What countries are already doing well certainly needs to be recognized and applauded, and support for these functions continued. As the medium and low performers in our study pursue robust implementation, they will need to think strategically about how to best allocate their limited attention and resources. The traditional core areas of immunization programmes—management, service delivery strategy and demand—cannot be taken for granted. These functions must remain at the centre of attention if the medium and low performers are to achieve the sustained success of the high performers.
Third, the considerable heterogeneity in measures of implementation across functions and countries offers a rich landscape for further research. For example, why were some countries more successful than others in the face of common management problems, such as human resource and transport difficulties? What is the optimal balance between different delivery strategies, such as outreach and services provided from fixed sites? It is difficult to interpret what appears to be the lack of a clear relationship between coverage and demand-creation activities in this sample. Others have found that coverage rates in low- and middle-income countries are, in general, more a function of supply-side than demand-side effects (Gauri and Khaleghian 2002). The relationship between coverage and the degree of external financing (larger in Ghana, Malawi and Ethiopia; smaller in Rwanda, Mauritania and Cameroon), and the presence of a SWAp or SWAp-like arrangement (present in Ghana, Malawi and Ethiopia; absent in Rwanda, Mauritania and Cameroon) is also unclear.
Finally, although our study was concerned primarily with describing programme implementation, and exploring the relationship between implementation and coverage, our data suggest that a range of factors outside the programme's control may also be important determinants of coverage variation. The Rwandan and Mauritanian experiences are particularly interesting cases of responses to contextual imperatives. Gauri and Khaleghian (2002) found a relationship between coverage rates in low- and middle-income countries and the quality of a country's institutions, its level of development and its degree of contact with international agencies. Global health partnerships are increasingly recognizing the important interaction between health system functioning and programme performance (Brugha et al. 2002; GAVI 2004), and substantial new resources are being committed to strengthening and helping us understand this relationship better (GAVI 2005).
Our study had several important limitations with respect to defining the constructs, applying the measures of implementation and generating hypotheses. First, although we felt confident that our five core functions provided a comprehensive picture of programme implementation, some turned out to be more operationally robust (governance, management, strategy) than others (financing and demand). This phenomenon was a direct function of and a common risk associated with the inductive, ‘insider’ point of view we adopted, in which we allowed informants and the historical record to define the functions, rather than outside investigators. In any case, this raises questions about the content validity of our study, both with respect to the relative degree to which each set of measures accurately tapped its respective function, as well as to the preliminary hypotheses that we drew from these data. Second, some of our methods (e.g. data collection and expert judgement) are probably more easily reproducible than others (e.g. case study composition). Third, our preliminary and retrospective application of the measures was susceptible to measurement bias. Key informants, field investigators and narrative analysts all knew in advance the performance status of each country; consequently, we cannot be absolutely certain that the strength of the measures in evidence in each country as judged by our panel was responsible for the different outcomes, or whether knowledge of the outcomes might have influenced the strength attributed to the measures. Fourth, our outcome of interest—coverage—is only one indicator of success in immunization (equity, efficiency and sustainability all matter), and the WHO/UNICEF Estimates have limitations (they may not always reflect the precise coverage reality of a country in any given year, and the magnitude of the uncertainty associated with these estimates is unknown as they are based on multiple data sources).
Finally, we encountered several analytical challenges. For example, when scoring the measures, analysts often had to weigh both quantitative and qualitative information in coming to a single decision, and ‘period estimates’ of implementation, in some instances, may have masked possible inter-annual variation in implementation. Analysts also had some difficulty disentangling demand-creation activities from outreach service delivery activities, which may have resulted in under-reporting of the former. They also encountered problems in valuing certain scores for the measures (i.e. is a significant level of external financing or the presence of a SWAp a positive or negative attribute?). Finally, the study was time and resource intensive.
|
Conclusion/future directions |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusion/future directionsEndnotesReferences |
|---|
This study represents a single, modest, but important contribution to what should be a broader research agenda to develop a robust, grounded theory of why some immunization programmes are more successful than others. The constructs and hypotheses generated by this research should be interpreted cautiously due to certain methodological limitations. Similar formative studies therefore are needed to validate the tentative constructs and hypotheses generated by our study, and to generate new hypotheses, all of which should be subjected to testing through other research designs and methods. Further empirical support for our findings would increase our confidence in the policy implications for national- and global-level decision-makers that we see emerging from our initial work in this area.
First, continuing innovation in immunization in areas such as new vaccine introduction and financing needs to be balanced against the need to address the unfinished agenda of routine immunization in sub-Saharan Africa. Despite improvements in recent years, coverage trends suggest that there is still considerable room for improvement in many sub-Saharan countries.
Second, examining and comparing how programmes are actually being implemented, in different settings, under different conditions, adds value to the normative, frequently prescriptive, guidance traditionally offered to countries. Our findings suggest that there are different paths to success and that not only what countries do, but also how they execute their programmes, seem to make a difference in coverage outcomes. More investment in systematic inquiry of this kind, and knowledge sharing among countries about how they execute and to what effect, may provide the impetus for many more countries to achieve higher levels of performance.
The six countries in this study recommended that this kind of highly participatory investigation be applied within countries to explore why some districts are more successful than others and to help every district become a high performer. The findings from such investigations would complement the ‘Reaching Every District’ (RED) initiative currently being promoted by WHO in Africa (WHO 2005). It would be instructive to note the extent to which the mix of possible interventions, both programme-specific and extramural, and the emphasis each should be given, may vary from the RED norm. Considering the inadequate progress being made towards reaching the child mortality reduction goal in Africa, more between- and within-country theory building about why some programmes perform better than others seems urgently needed.
Third, study findings confirm what many others have observed: that the solutions to overcoming the challenge of sustained coverage do not necessarily reside within reach of the immunization programme alone. Contextual challenges and opportunities in the larger health system, if properly addressed, could enhance the more direct efforts undertaken by immunization programmes to improve their performance.
Several steps can be taken to improve upon our methods. More intensive probing of respondents than was possible in the present study and expanding the pool of informants to include those with expertise in underdeveloped constructs, such as financing, demand generation and a range of contextual factors, would improve content validity. Precise documentation of all technical aspects of fieldwork, including data collection procedures and analysis protocols leading to case study development, would improve the chances of reproducibility of the findings. Measurement bias can be improved through several measures: by increased training of jurors in how to deal with likely ambiguities in the data and with data highly susceptible to alternative interpretation, and in how to value scores for problematical measures; by increased inter-juror reliability testing; and by blinding the jurors to the performance status of each country they are asked to rate. Time and resources permitting, a supplementary panel could be enlisted to validate the measurements of the primary panel. Ensuring adequate funding for all aspects of the research, and increasing the number of both field investigators and panel jurors, from the outset, can greatly reduce the time necessary for completing this kind of study.
Finally, although the literature on improving immunization coverage in all settings has focused primarily on discrete interventions to improve the supply of, demand for and access to services, there has been little work that acknowledges immunization as a dynamic system, and which attempts to examine the contribution of a programme's related parts, and the contextual factors in its environment, to improving coverage. This is an innovative design feature of this study, which we believe should be considered by future investigators.
|
Acknowledgements |
|---|
We thank the Ministries of Health and the WHO and UNICEF offices in the six countries in which this study was conducted for their support and active participation. We acknowledge the significant contributions to various aspects of this study of Bank colleagues Rashmi Sharma, S H Thilsted, Anthony Measham, Oscar Picazo, Deepti Tanuku and Robin Martz. Ed Bos, Logan Brenzel and Peyvand Khaleghian, also from the World Bank, provided valuable advice and technical inputs. We thank Jennifer Bryce, Michael Reich and Stephen Hadler for their comments on an earlier draft of this paper. We thank Alex Rowe, Vance Dietz, Lisa Cairns, Bob Keegan, Mary Harvey, Maureen Birmingham, Mercy Ahun, Michael Favin, Paul Fife, Rebecca Fields, Robert Steinglass and Daniel Kress for their comments on an earlier working paper. The study would not have been possible without the encouragement, advice and support of Amie Batson and Ok Pannenborg of the World Bank. This work was supported with financial assistance from the Child's Vaccine Program at PATH and the government of the Netherlands, through the World Bank-Netherlands Partnership Program (BNPP). The findings, interpretations and conclusions expressed in this article are entirely those of the authors and do not represent the views of the World Bank, its Executive Directors, the countries they represent, the Centers for Disease Control and Prevention, or the afore-mentioned colleagues.
|
Endnotes |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusion/future directionsEndnotesReferences |
|---|
1 The ‘official country estimate’ is left to the discretion of each country and reported to WHO by the government in the annual Joint Reporting Form (JRF). The estimate may be based on standard administrative data, or the latest available survey data, or an adjustment of either (personal communication, WHO).
2 Statistical Analysis Software (SAS) was used to calculate a slope of the linear regression line that fitted to the data points for each country using the method of least squares.
3 Data from 2003 and 2004 (Table 4) indicate that Cameroon has made important progress in immunization coverage.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusion/future directionsEndnotesReferences |
|---|
Bos E, Batson A. Using immunization coverage rates for monitoring health sector performance: measurement and interpretation issues (2000) Washington, DC: Health, Nutrition, Population/The World Bank. HNP Discussion Series.
Briss PA, Rodewald LE, Hinman AR, et al. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. The Task Force on Community Preventive Services. American Journal of Preventive Medicine (2000) 18(Suppl.):97–140.[Web of Science][Medline]
Brugha R, Starling M, Walt G. GAVI, the first steps: lessons for the Global fund. The Lancet (2002) 359:435–38.
Bryce J, Terreri N, Victora CG, et al. Countdown to 2015: tracking intervention coverage for child survival. The Lancet (2006) 368:1067–76.
Bryce J, Victora CG. Child survival: countdown to 2015. The Lancet (2005) 365:2153–4.
Camp R, Tweet A. Benchmarking applied to health care. Journal on Quality Improvement (1994) 20:229–38.
Chee G, Fields R, Hsi N, Schott W. Evaluation of GAVI immunization services support funding (2004) Bethesda, MD: Abt Associates.
Claeson M, Waldman RJ. The evolution of child health programs in developing countries: from targeting diseases to targeting people. Bulletin of the World Health Organization (2000) 78:1234–45.[Web of Science][Medline]
Gauri V, Khaleghian P. Immunization in developing countries: its political and organizational determinants (2002) Washington, DC: The World Bank.
GAVI. Alleviating system-wide barriers to immunization: issues and conclusions from the second GAVI consultation with country representatives and global partners (2004) Geneva: Global Alliance for Vaccines and Immunization.
GAVI. New Delhi GAVI Alliance Board Meeting: Final summary report (2005) Geneva: GAVI Alliance.
Hardon A, Blume S. Shifts in global immunisation goals (1984–2004): unfinished agendas and mixed results. Social Science and Medicine (2005) 60:345–56.[CrossRef][Medline]
Marsh DR, Schroeder DG, Dearden KA, Sternin J, Sternin M. The power of positive deviance. British Medical Journal (2004) 329:1177–9.
Martin JF, Marshall J. New tendencies and strategies in international immunization: GAVI and the Vaccine Fund. Vaccine (2003) 21:587–92.[CrossRef][Web of Science][Medline]
Muraskin W. The Global Alliance for Vaccines and Immunization: is it a new model for effective public-private cooperation in international public health? American Journal of Public Health (2004) 94:1922–25.
Otten M, Kezaala R, Fall A, et al. Public health impact of accelerated measles control in the WHO African Region 2000–03. The Lancet (2005) 366:832–39.
Pegurri E, Fox-Rushby JA, Walker D. The effects and costs of expanding coverage of immunization services in developing countries: a systematic literature review. Vaccine (2005) 23:1624–35.[CrossRef][Web of Science][Medline]
Phillips MM. G-8 nations shape plan to fight diseases: goal is getting drug makers to generate vaccines against illnesses in developing world. The Wall Street Journal (2006) p. 8. February 13.
United Nations. The Millennium Development Goals Report, 2005 (2005) New York: United Nations.
Wagstaff A, Claeson M. The Millennium Development Goals for Health: Rising to the challenges (2004) Washington, DC: The World Bank.
WHO. The World Health Report 2005: Make every mother and child count (2005) Geneva: World Health Organization.
WHO/UNICEF. Vaccines, Immunization and Biologicals, WHO/UNICEF Review of national immunization coverage from 1980 (2005) Online at: http://www.who.int/vaccines/globalsummary/timeseries/tswucoveragedtp3.htm.
Accepted for publication 3 June 2008.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||