Health Policy and Planning Advance Access originally published online on July 29, 2008
Health Policy and Planning 2008 23(5):351-360; doi:10.1093/heapol/czn026
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
This article appears in the following Health Policy and Planning issue: Future directions for health policy analysis: a tribute to the work of Professor Gill Walt [View the issue table of contents]
Addressing the theoretical, practical and ethical challenges inherent in prospective health policy analysis
Overseas Development Institute, 111 Westminster Bridge Road, London, SE1 7JD, UK. Tel: +44 (0)207–922–0300. Fax: +44 (0)207–922–0399
E-mail k.buse{at}odi.org.uk
 |
Abstract |
|---|
 Top Abstract Introduction: the case for...Supporting advocacy coalitions...Doing research on the...DiscussionConclusionsEndnotesReferences |
|---|
As a function of the inherently political nature of health policy, there have long been calls for, as well as guidance on, analysis of its political dimensions to inform practice. Yet there are few accounts in the literature of systematic attention to real-time documentation and analysis of political-economy factors and feedback to engender reform. The dearth of such prospective policy analysis is perhaps understandable given the many intrinsic difficulties in such an enterprise. This paper provides an outline approach of how researchers might work together with advocacy coalitions (or other political actors) to document and analyse the efforts of such coalitions to use policy analysis to influence the policy processes—agenda setting, policy formulation and policy implementation—in which they engage. In so doing, it identifies challenges based on reviews of the theoretical, methodological and empirical literature as well as the experience of the author. The aim of the paper is to generate debate to assist in resolving the myriad challenges inherent in prospective policy analysis. The paper responds to appeals for political research which addresses the problems confronting political actors so as to guide future action-research for evidence-informed, pro-poor health policy.
Key Words: Health policy, policy analysis, policy process, prospective analysis, action-research, policy networks, advocacy coalitions
KEY MESSAGES
|
|
Introduction: the case for prospective policy research |
|---|
|
TopAbstractIntroduction: the case for...Supporting advocacy coalitions...Doing research on the...DiscussionConclusionsEndnotesReferences |
|---|
As a function of the inherently political nature of health policy-making, there have long been calls for, as well as guidance on, analysis of its political dimensions (Thomas and Grindle 1994
; Walt and Gilson 1994; Reich 1995a,b; Leighton 1996; Collins et al. 2002; Gilson et al. 2003; Kajula et al. 2004; Thomas and Gilson 2004). Michael R Reich has been most explicit in advocating for political analysis of health policy and has co-developed software for use by pro-reformers to better understand the interests, influence and positions of actors in relation to specific policy proposals (Reich and Cooper 2001). The Reich-Cooper software also assists in identifying a range of strategies and tactics which can be adopted by reformers to change the political landscape by addressing the power and position of key players as well as their perceptions of the problem and of the policy solution—which is further elaborated in ‘Getting Health Reform Right’ (Roberts et al. 2004).
Despite calls for pro-actively managing the political dimensions of health sector reform, and some retrospective analysis of attempts at doing so, there are few accounts in the literature of systematic attention to political-economy factors, forward looking, real-time documentation, immediate lesson learning (analysis), and feed-back to engender strategies to engage in policy processes—‘prospective policy analysis’ for want of a better term. Glassman (1999) provides one partial account in relation to health reform in the Dominican Republic which met with limited success.1 There are undoubtedly many undocumented accounts of efforts by stakeholders to better understand the political obstacles and opportunities confronting their policy aims so as to devise strategies to improve the chances of their policy success.2 Moreover, there is increasing interest in undertaking such analysis to inform action, even by mainstream aid agencies such as the United Nation's Population Fund (UNFPA) through its Applied Interest Group Analysis (O’Brien et al. 2006) and the United Kingdom's Department for International Development (DFID) in sector-level ‘Drivers of Change’ work. Nonetheless, it would appear that prospective policy analysis remains an under-explored, under-analysed and under-reported area of health development, as discussed below.
The first generation of reforms in developing countries, concerned largely with economic stabilization and structural adjustment (e.g. tariff reductions, exchange rate devaluation), were often referred to as ‘stroke-of-the-pen’ reforms because they were often single-event policy adjustments and self-executing. In contrast to these macro-economic policy changes, Joan Nelson (1999) argues that: ‘Social sector reforms are a different ball game, with far more actors, less leverage, different fields of play, a much longer playing period (with unpredictable time-outs) and uncertain scoring’. To use Nelson's sporting analogy, is prospective (as opposed to retrospective) health policy analysis to academics like cricket is to North Americans—the timeframe far too long (a policy process might take a few years to unfold) and the play far too intricate and complicated to warrant ongoing attention?
Moreover, literature on the political economy of social sector reform suggests that the prospects for pro-poor reform are not encouraging (Batley 2004). Rational choice theory explains these outcomes by drawing attention to the asymmetries of power and incentives facing winners, losers and politicians (Olson 1974)—specifically that the potential losers are aware of their losses and quick to respond (e.g. health provider unions) while potential winners (e.g. primary beneficiaries of interventions) are less likely aware of intended benefits (which may not accrue immediately) and are less well organized. Nelson (2000) draws attention to the strong administrative and professional apparatuses that resist reform in these sectors. Political economists have focused on the role of formal and informal institutions in explaining reform failure (North 1991; van de Walle 2001). Institutional factors affect the extent to which winners and losers have access to decision-making forums, the value of their political assets, among other political variables (Swank 2002). Interests defending the status quo tend to be more powerful than reformers as they are usually the winners of prior policy contests and have rigged the rules of the game in their favour (Oliver 2006)—the so-called ‘mobilization of bias’ (Schattschnieder 1960) in one literature and path-dependency in another. It may be that academic observers shun prospective analysis for fear of being left with a ‘non-event’ to explain (i.e. no change in policy) or at least a ‘no-score’, although it may be the case that there are other reasons why academics have not turned their attention to such work.3
Yet progressive social policy reform has been possible in some Latin American countries ‘despite the odds’ identified above (Grindle 2002). Grindle does not deny the role of interests and institutions in the success, limited success, or failure of reform. However, the cases she analyses point to the importance of the policy process, the considerable agency and choices facing reform leaders and change teams (technical officers brought together from various agencies with a mandate to generate political strategies for a reform), and the use of different strategies (e.g. careful timing, creation of networks, use of information or symbolic elements to mobilize public opinion, changing venue, etc.) in explaining outcomes. Nelson's overview of social sector reforms outlines the different tactics that can be deployed in agenda setting, getting executive approval, getting legislative and public acceptance, and launching and sustaining reform in response to the distinct political challenges at each stage of the policy process (Nelson 2000). Gonzalez-Rosetti and Bossert's analysis of relative success in Chile and Colombia is compared with Mexican reforms and highlights the strategies of the reform team, revealing some conditions for success, but also underlining the existence of strong state-society networks buttressing the power of pro- and anti-reform groups (Gonzalez-Rosetti and Bossert 2000). Hence, despite the power of interests and institutions to resist change, reformers have ‘room for manoeuvre’ to deliberately bring about changes in public policy (Grindle and Thomas 1991). So the pro-reform game is possible, particularly with sensible coaching; coaching which could be provided by policy analysts with the luxury of stepping back from the game, but this seems the exception rather than the rule.
Eugene Bardach (2006) concurs with social sector analysts such as Grindle and Reich, that all else being equal, while interests, institutions and even ideas (due to the mobilization of bias) may be stacked against pro-reformers, success will be easier to achieve if pro-reform groups are prepared when ‘opportunities’ present themselves (Kingdon 1984). To be prepared entails some understanding of the political dimensions of the policy in question, as is made possible by prospective policy analysis.
This paper advocates two interlinked processes. First, prospective policy analysis in support of concurrent health policy-making (agenda setting, policy formulation or implementation). The policy analysis proposed itself builds on frameworks developed by other analysts, particularly Reich's policy ‘content’ and ‘actor’-centric approach (Reich 1995a,b), Walt and Gilson's heuristic which brings policy ‘process’ and ‘context’ into view (Walt and Gilson 1994), alongside broader models which incorporate the roles of ideas and discourse in policy, such as Peter John who views policy as the ongoing interaction of institutions, interests and ideas (John 1998). The paper further advocates that such analysis is undertaken by members of nationally based networks (specifically advocacy coalitions)4 with pro-poor aims, collaboratively with analysts (ideally by national analysts where capacity exists and/or expatriates such as myself) where external support may be viewed by such groups as desirable and would add value given their interests and capacity in policy analysis.5 The purpose of the analysis is to equip the networks with information to develop strategies and tactics to engage more effectively in the policy process. Here too I borrow from Reich's ‘PolicyMaker’ tool, but extend its range to incorporate policy context, process and discourse variables.
Secondly, the paper argues for research on the impact that structured and prospective approaches to policy analysis may have on policy processes. As such, it proposes research which sheds light on: (1) how the networks use policy analysis tools, (2) the understanding that those groups gain from such policy analysis, for example of the policy environment, including sources of power as well as the perceptions and tactics of counter-reformers, (3) how the groups use that knowledge to craft strategies and tactics to influence policy change, and (4) whether or not those strategies are effective.6 The proposed research will be of interest to participating networks themselves and, even if not strictly generalizable, to other groups working on other policy projects. Real-time learning to distil lessons on how the barriers to policy reform can be addressed and to inform reformist political strategies represents an under-researched and potentially rich field of enquiry (Campos and Syquia 2006). Moreover, it contributes to the development of a stronger knowledge base in health policy analysis as called for by Gilson and Raphaely (2008, this issue).
This paper proceeds in three sections. First, there is discussion of the rationale for greater interaction between policy analysts and advocacy coalitions. Second, the paper proposes an outline approach of what this action-research might look like in practice and what hypotheses it might test. Third, it identifies some of the theoretical, practical and ethical challenges inherent in such research and contributes to a discussion of possible solutions based on relevant literature and the experience of the author. The aim of the paper is to generate debate to assist in resolving the myriad challenges inherent in prospective policy analysis. Consequently, the paper responds to Shapiro's (2005) appeal for political research which addresses the problems confronting political actors so as to guide future action-research for evidence-informed, pro-poor health policy reform.
|
Supporting advocacy coalitions to do policy research |
|---|
|
TopAbstractIntroduction: the case for...Supporting advocacy coalitions...Doing research on the...DiscussionConclusionsEndnotesReferences |
|---|
Considerable research has been undertaken on the characteristics of networks, and the environments in which they operate, to understand what makes them effective in influencing policy (Marsh and Smith 2000
). While quite distinct forms of networks have been identified, two ideal types are typically distinguished. A ‘policy community’ is a relatively small network, with a high degree of integration among members who are significantly dependent upon one another, which restricts access to non-members. Policy communities tend to be very cohesive. An ‘issue network’, by contrast, has relatively more members, less dependence of members upon one another and a looser organizational form. One form of policy community, the advocacy coalition, has received considerable attention for its role in the policy process. Advocacy coalitions are distinguished by a shared set of norms, beliefs and policy goals. They can include politicians, civil servants, members of interest groups, journalists and academics, among others (Sabatier and Jenkins-Smith 1993). Advocacy coalitions are potent forces within the policy rubric which can serve to limit participation, decide which issues will feature on agendas, shape the behaviour of participants, privilege the interests of certain groups, or manipulate the evidence reaching decision-makers (Rhodes 2006). Nelson (2004), Gonzalez-Rosetti and Bossert (2000), among others, have commented on the existence of strong state-society networks, having many of the features of advocacy coalitions, buttressing the power of pro- and anti-reform groups in the health sector. The potential of such networks to influence policy motivates my interest in working with them to support government officials with policy analytic models to better manage discrete policy reforms.
Given the imperative to advance evidence-informed policy to bring about public health outcomes,7 networks are ideally situated to provide a powerful conduit for evidence to reach decision-makers. Conceptualizing policy as a complex interaction of institutions, interests and ideas (John 1998) leads one to consider the relationship between evidence and policy as a messy and indirect one which may take many forms (Weiss 1979). Notwithstanding this indirect relationship, the elective affinity model holds that a decision-maker is more likely to react favourably to research findings if s/he has participated in the research process in some way, if the findings are timely in the decision-making process, and coincide with the values and beliefs of the policy audience (Short 1997). This might explain why in some contexts, advocacy coalitions—which bring researchers, advocates, managers and others in direct contact with decision-makers—have been successful in achieving evidence-informed policy (Gibson 2003). Building on these findings and recent thinking on the research-to-practice linkages, clearly more can be done to improve the knowledge transfer between researchers and decision-makers (Lavis et al. 2003). The ‘deliberative processes’ of engaging technical, scientific experts, programme managers, decision-makers and the public to collectively weigh various types of evidence (scientific evidence from different disciplines and experts as well as colloquial evidence on values and perceptions etc.), in collaborative analysis of policy options seems potentially rewarding (Lomas et al. 2005; Thompson et al. 2006). The deliberative approach may embody other valuable characteristics as it ‘focuses on the crucial role of language, rhetorical argument, and stories in framing debate ... It also brings in the local knowledge of citizens – both empirical and normative – relevant to the social context to which policy is applied’ (Fisher 2003). Decision-makers will not only benefit from greater proximity to technical experts but also from better evidence concerning a range of policy relevant variables which can inform the tactics they employ to bring about change.
For their part, health scientists, activist groups and health programme managers should be concerned about the political feasibility of the policies which they pursue, for if they pursue a goal which is not politically feasible the opportunity costs are often high, including disillusionment and foregoing more attainable goals (Galston 2006).
Consequently, when requested, policy researchers could valuably work with members of advocacy coalitions to assist them to better understand the political dimensions of the reforms they wish to see enacted and to use that information to devise strategies and tactics for so doing (Buse et al. 2006). One approach to doing so is outlined in the following section.
|
Doing research on the use of policy analysis by advocacy coalitions |
|---|
|
TopAbstractIntroduction: the case for...Supporting advocacy coalitions...Doing research on the...DiscussionConclusionsEndnotesReferences |
|---|
Research on prospective policy analysis must be ‘piggy-backed’ and ‘demand-led’; that is, the research relies on studying an already unfolding policy process and requires that the researchers are invited by country-level actors involved in that policy process who may wish to take a more structured approach to the political analysis, the resulting policy engagement and the proposed research on it. The research is not an assessment of the efficacy of technical interventions, but rather of the effectiveness of different strategies and tactics for intervening in the policy process. Consequently, the purpose of the research is two-fold: (1) to assist an advocacy coalition in being more effective in managing the political dimensions of the policy process; and (2) to study which strategies and tactics are more effective in changing the political dimensions of the policy process.
Policy researchers, be they national or expatriate, invited into such policy processes would likely choose to work on technically appropriate, globally accepted, evidence-informed tracer policy intervention(s) which contribute to public health goals, to the extent that such an ideal is feasible.8 The policy researcher(s) would work alongside a collaborator, who is both a member of a pre-existing advocacy coalition (or issue network) and has interest, experience and/or capacity in policy analysis9—a policy research ‘broker’—called here collectively, the research team. The research team would discuss issues concerning methods of policy analysis with members of the network. The research team would also learn together with network members while developing and implementing policy engagement strategies over a number of years. The researchers would document the activities and the impact (the limits of attribution are discussed later) that these strategies appear to provoke in the policy arena over the course of the intervention (e.g. in relation to resource allocation, technical recommendations, media coverage, attitudes of key stakeholders). The team would continuously update the policy analysis (indeed a major benefit of doing real-time analysis is the opportunity it provides to link theories, analytical frameworks and data collection, and to change these if alternatives appear more promising in effecting change). The results of the analysis would be periodically reviewed by the advocacy coalition so as to update and revise the policy engagement strategies. Decisions and activities would be documented by the researchers. The details of the proposed approach are enumerated in Box 1.
| Box 1 Approaches for implementing a Policy Influencing Framework A Policy Influencing Framework which consists of two distinct activities:
The approach to policy analysis is based on a variety of authors (Hogwood and Gunn 1984
Based on the policy analysis, the political feasibility of the proposed intervention will be assessed. The analysis will also be used to develop policy engagement strategies for intervening in the policy process with respect to the tracer policy, based again on relevant lessons from the literature (e.g. Roberts et al. 2004
|
The research, which would piggy-back on the efforts of the coalition to implement a policy-influencing strategy, could test a number of hypotheses though the steps outlined in Box 2. Broad hypotheses which would need to be refined based on the specific policy and the context within it is being pursued include:
- More systematic approaches to the collection and analysis of policy information (e.g. concerning interests, institutions, ideas, and policy processes and context) can provide advocates with the knowledge to make an initial assessment of the prospects that the policy will be implemented.
- More systematic approaches to the analysis of policy information can alter the balance of power between advocates and anti-reformers and enable advocates to intervene more effectively in the policy process.
- More systematic approaches to analysis of policy information can shift the positions of the players involved in the policy process. In particular, bargains can be made by advocates with those that are neutral or opposed to render them more supportive or less opposed by altering the content of policy, horse trading, making promises and threats or using other strategies.
- Systematic approaches to analysis of policy information can alter the number of players involved in the issue. This could involve devising strategies to mobilize those groups which are neutral and to demobilize opposed groups or shift the venue of policy discussion.
- Systematic approaches to analysis of policy information can assist advocacy groups to alter perceptions of the problem and solutions to the policy issue. This might include devising strategies involving data, arguments, emotive appeals, emphasizing ‘doability’ and feasibility of the intervention, and/or mobilizing public opinion while challenging the arguments of the anti-reformers.
| Box 2 Cross-sectional policy analysis to test the research hypotheses The research hypotheses will be tested through an initial and end-point analysis of the:
|
|
Discussion |
|---|
|
TopAbstractIntroduction: the case for...Supporting advocacy coalitions...Doing research on the...DiscussionConclusionsEndnotesReferences |
|---|
The preceding description of one approach to prospective policy analysis throws up many unanswered questions as well as some obvious challenges which may account for the dearth of published accounts of its practice. Some of the difficulties inherent in the enterprise are elaborated, alongside a discussion of how they may be addressed.
Theoretical and methodological challenges to prospective policy analysis
Both the analysis undertaken by the advocacy coalition as well as the accompanying research on the prospective analysis and policy engagement throw up questions as to the selection of appropriate theoretical frameworks and corresponding causal relationships they describe. The theoretical underpinnings of much policy analysis are diffuse and problematic when applied in the complexity of the real world. Despite attempts by some social scientists to apply the positivist paradigm of the natural scientists to increase knowledge concerning general, causal and decontextualized relationships in the social and political sciences (e.g. King et al. 1994), emulating the natural sciences will remain a challenge in the policy field. In the natural sciences, causality refers to an account that explains an outcome through a measurable variable that exists independently of any participant/observer's understanding. Yet, in the policy sciences, the search for explanation is intimately bound up in perceptions held both by the participant and the observer of the variable (which may or may not even account for the occurrence). Klein and Marmor (2006), for example, characterize policy-making as ‘a strange theatre’, an understanding of which requires the researcher to enter the ‘assumptive worlds’ of the policy-makers—that is, their mental models including their interpretations of the environment and their prescriptions as to how it should be structured. The assumptive worlds of policy-makers, influencers and implementers are all important in the social construction of the problem and the policy. Interpretive theories attempt to account for why an actor behaves in a particular way and results in understanding that is both subjective and context specific. This is far from universal and generalizable knowledge generated in the natural sciences but useful in and of itself in drawing attention to factors and variables which have been perceived—by the researcher and/or subject—to be associated with, if not having accounted for, change in a particular setting (based on a particular experience and interpretation of history).
Beyond the problem of the ‘independent’ or objectively verifiable variable is the problem of the laboratory setting; the social world is not generally amenable to the approach of the natural sciences where basic research proceeds by fragmenting the world into abstract, analytical slices which are then studied individually. Etzioni (2006) argues ‘policy research at its best encompasses all the major facets of the social phenomenon it is trying to deal with’. Not only are there generally too many ‘confounding variables’ to control, even these variables remain difficult to pin down [think of the range of possible variables subsumed under the broad rubric of ‘institutions, interests and ideas’ which, according to John (1998), interact to create policy]. Immergut's analysis of the literature on institutional constraints on policy, for example, suggests that the institutions typically studied do not conform to any ideal type (Immergut 2006).
There is similar trouble with the unit of analysis itself (i.e. policy—the dependent variable). While it may be relatively straightforward to look for the determinants of policy as conceived at the level of principles (e.g. DOTS as a general principle of ensuring drug treatments are administered under observation), policy lines (i.e. the more specific measures such as 6 months of drug therapy observed by a health worker), or measures (e.g. specific guidelines and budgetary allocations associated with DOTS) (Page 2006), or even as bundles of decisions, we all know that policy is a more elusive concept. Consider Bachrach and Baratz (1962) who argue that it is worth considering those issues that never make it on to the agenda (the ‘non-decision’) or Lukes (1974) who describes some policy as thought control. So the policy outcomes observed in policy processes may not entail a change in legislation, a policy document or technical guidelines, but may be as subtle as a shift in perceptions of an issue that sets the stage for more apparent and tangible change at a future date. The subtle and hidden manifestations of policy are very difficult to discern, observe and measure, and are, therefore, tricky to study.
The foregoing alludes to the importance of power (whether as manifest in institutions or interest groups) in the policy process as well as the importance of ideas; indeed for Majone (1989), ‘analysis of public policy, like policy making itself, is an exercise in persuasion’. This takes us back, on the one hand, to the issue of the objectivity of policy research and, on the other, to the links between ideas and interests. We are reminded by Anthony Giddens (1979) that agency shapes structures, which in turn condition agency and, consequently, we are faced with the challenge of how best to meaningfully disentangle the variables inherent in power, institutions and ideas for their analysis.
Even where policy theories emerge from some empirical test as robust, experience suggests that they do not travel particularly well, drawing attention to the problem of context. For example, Kingdon's powerful approach to understanding agenda setting arose from an analysis of the pluralistic and fragmented political life of the federal American system (Kingdon 1984)—a system with different dynamics than those present in the core-executive system of the UK let alone the neo-patrimonial political systems entrenched in much of sub-Saharan Africa, but which has been employed usefully in a range of contexts in practice. Much the same has been said of the Advocacy Coalition Framework (Parsons 1995) which I find attractive in guiding my understanding of policy change.
A further concern relates to the dearth of theory in so far as some aspects of the prospective action components of the approach are concerned. While institutionalized authority, interest group power, and inter-personal influence have been demonstrated to have been associated with specific policy outcomes in particular settings, the literature on process-related tactical approaches to policy influencing is less theorized.
The explanatory and predictive ability of policy research is further limited by the role that chance plays in the policy process. Downs’ description of the ‘issue attention cycle’ goes so far as to argue that a dramatic event is the decisive factor in putting an issue on a policy agenda (i.e. something beyond the control of a skilled policy activist) (Downs 1972). Personally I would be wary about characterizing any factor as the decisive one, but the point is that these exogenous events may be clear to see in retrospect, but are by definition exceedingly difficult to predict in advance, let alone engineer.
This brief foray into the realm of the theory of policy analysis leads me to take a very cautious approach to defining ex ante the theoretical basis of prospective policy analysis.10 I conclude that identifying one particular theory which is likely to be directly relevant to explaining change or in predicting outcomes in a specific setting would be rather challenging. A more fruitful approach is to draw, eclectically, upon many theories developed by different disciplines (political economy, economics, sociology, anthropology, organizational management, history—to reveal the fabric of institutions or the hidden interests or give meaning to assumptive worlds meaning to assumptive worlds, and so on) which provide guidance on the selection of variables which can reasonably be expected to have relevance (to institutions, interests and ideas) and which will inform research strategies, methods and interpretation.
The notion of ‘reasonably expected to be relevant’ is supported by some literature. Policy analysis has been likened to a ‘mood’ more than a science, ‘a loosely organized body of precepts and positions’ (Goodin et al. 2006). Bardach (1972) argued that applying a social scientific standard in policy analysis that can offer an satisfying ‘explanation’ is elusive. He proposes in the place of explanatory power, the creation of a practical standard, ‘an intertemporal map of the foreseeable risks and opportunities that might emerge’; the purpose of the map is both to identify possible variables11 to monitor and analyse, and to develop prospective strategies to engage in the policy process. Similarly, the prospective action-research which I describe is not concerned with validating general causal theories but rather addressing problems in the real world as experienced by political actors (specific ones in this context, who wish to advance an evidence-informed project), on the one hand, as a means to a political end and, on the other, as a means to develop and test theories of the possibilities of politics in a given context which may provide guidance to other activists and researchers working in different contexts. In so doing, it would also add to the small but growing literature which attempts to explain health policy change.
In much the same way that prospective policy analysis would rely on multiple disciplines to inform the identification of important variables so too does it rely on various methods to elicit policy-relevant data. Indeed, one strength of the approach is the ability of the researchers, on the basis of reflection, to change the theories, analytical frameworks and the nature of data collected if considered useful. Given the highly subjective nature of the field and problems arising from the insider-outsider interface (see Walt et al. 2008, this volume), robust analysis would require high levels of triangulation and reflexivity, as well as other methodological checks which require further discussion beyond what is possible here.
Practical challenges to prospective policy analysis
The practical challenges to prospective policy analysis are many and varied.12 They include, for example, how to develop capacity within networks to undertake policy analysis. While ongoing training and assistance from a supportive policy analyst may constitute one approach, linking the research with a larger and coordinated effort to develop a cohort of research brokers may be more strategic and sustainable. A second issue is how to identify issue-specific networks with whom policy researchers may work. Four ‘ideal’ preconditions for network selection include those which: (1) pursue pro-poor health goals through evidence-informed interventions; (2) have an interest in improving their impact through better use of policy analysis tools; (3) have at least one member willing to act as a policy research broker; and (4) are comfortable with the efforts being documented and analysed by outside policy researchers (be they national or expatriate).
This leads to the question of how best to broker relationships and agreements among network members, research brokers, researchers and funding agencies (research or donor), and to the related question of longish-term financing (for both research and policy influencing strategies, both of which may require significant resources). Dealing first with the latter, does one wait for a call for such research from a research commissioning body (will this ever happenquest;) or does one proactively seek out donors who might be sympathetic to the need for more policy analysis (but have little interest in research or in funding the strategies and tactics—their possible ethical objections are treated below)? It appears that the researcher, in partnership with the network, must take on the entrepreneurial task of selling prospective policy analysis, hence this paper. External finance raises the question of how to allay fears that the sponsoring agency may co-opt the coalition for its own ends or the potential risk that privileged members of the coalition—the brokers—will use the influence that accrues from analysis for other ends. Indeed, analysis and the development of engagement tactics is likely to engender differences of opinion and rifts within the network which call into question the need to deal with the issue of trust. Clear ex ante specification of some research partnership ground rules would be helpful, but too many might undermine trust within the network, so a certain element of risk would have to be accepted. The risks are exacerbated by the long time horizons required for such research (i.e. 2 or 3 years), with members of the coalition shifting jobs (particularly problematic in the case of the public officials) and perhaps even interest in the policy project. But such is the fate of the policy researchers who, in the words of Bardach (2002), must prepare themselves for ‘a long campaign potentially involving many players, including the mass public’.
There are further practical challenges of obtaining, documenting and sharing detailed information on the strategies and tactics adopted (i.e. with pro-reformers while concealing them from anti-reformist elements), but there is too little space to consider them here.
Ethical concerns and red herrings regarding prospective policy analysis
Finally, the prospective policy analysis is beset by real ethical concerns and more spurious ones. As to the former, policy analysts come with their own values and ideological interests, and these will inform whether or not they decide to work with a particular government reform team, UN organization, or policy community on any particular policy issue and/or technical intervention. Even if the policy analyst did not support the specific approach, s/he may want to present the arguments against, to try to shift or alleviate policy positions. As noted above, my values and principles, which span utilitarianism (the ends justify the means) and liberal egalitarianism (justice as fairness) (Rawls 1971), would lead me to support groups which pursue pro-poor health goals through evidence-informed interventions. In some cases the evidence may not be sufficiently robust to make such judgements and it would then behove me to convince the advocacy coalition to work with researchers to generate convincing evidence (which in itself is a pre-requisite for improving the prospects of policy implementation).
A more thorny issue concerns transparency and confidentiality. Modern policy-making stresses the importance of transparency and procedural legitimacy (UK Cabinet Office 1999). Yet, much of the data upon which prospective policy analysis and action is based will be collected in an informal manner—during meetings in which the advocacy coalition members participate, in corridor discussions, and also in social settings. Often it would be imprudent to divulge the ‘hidden agenda’ behind seeking information on, for example, the underlying interests of a given stakeholder. It is, however, likely that an academic ethical committee would demand ex ante reassurances on informed consent as well as confidentiality or even anonymity of respondents, something that would make the exercise rather futile. One option is to eschew academia in this quest—for it does not have the final word on ethics—particularly where failure to develop political strategies and tactics to ensure the implementation of pro-poor policies will result in less cost-effective health investment. At present, it is not clear what alternative ethical bodies are available. A further ethical issue concerns how network members communicate the nature of their policy project to the wider policy community—and certainly communication will be one of the strategies since policy is so much about persuasion—but here we are dealing with how the coalition might characterize or represent its goals in a truthful manner while not jeopardizing its project.
Spurious ethical objections have been raised in response to the prospect that external groups support (with funds or policy analytic advice and support) domestic constituencies to improve the prospects for the development and implementation of evidence-informed, pro-poor policies, but that argument cannot be sustained. To begin with, the distinction between ‘external’ groups and internal issues is hard to uphold in this context. The main pro-poor policy goals now transcend national boundaries (e.g. the MDGs), for the good reason that achievement of those goals requires global efforts. Neither the drivers nor the blockers of the corresponding reforms are restricted to the domestic policy arena. Experience suggests that pro-poor reforms will likely be blocked by a variety of national and international groups with an interest in maintaining the status quo or in altering it in ways that may further their interests at the expense of the poor.
Secondly, the aim of prospective management of reform politics is to empower national groups (and build national capacity) to achieve reforms which have widespread technical acceptance (i.e. on the weight of economic and epidemiological evidence) and support, but which are faced with entrenched interests, either at home or abroad or both. In other words, the image of external actors foisting ill-considered policy measures on unwilling countries is entirely inappropriate. In any case, researchers (and funding agencies which support them) can minimize any risk of this sort by scrupulously supporting only coalitions that are attempting, as far as possible, to pursue technically appropriate, globally accepted, evidence-informed interventions, or are pursuing goals which would include the generation of such evidence concerning issues confronting the poor.
Indeed, the ethical boot is arguably on the other foot in that it is not ethical for the international community to agree to targets such as the MDGs without supporting groups within countries to attain them, not only with funding, but in dealing with the opposition that is put in their path.
|
Conclusions |
|---|
|
TopAbstractIntroduction: the case for...Supporting advocacy coalitions...Doing research on the...DiscussionConclusionsEndnotesReferences |
|---|
Some of the political dimensions of policy change may be readily apparent, particularly in retrospect, but to elucidate the likely politics ex ante is challenging, to know how to best intervene without policy analysis is even more challenging, and to assess the impact of the interventions on the basis of such analysis is more tricky yet. Nonetheless, I would argue that the future prospects of the implementation of many evidence-informed interventions will be in jeopardy if such challenges are not confronted. While the match may be difficult and complex, the odds perennially stacked against the underdogs and the rules governing the involvement of outside analysts uncertain, it is time to move this from a spectator sport to one in which policy analysts demonstrate a willingness to try out a new playing field, i.e. one which is prospective.
|
Acknowledgements |
|---|
I would like to thank the participants at the Health Policy Analysis Workshop held in London on 21–22 May 2007, and in particular Lucy Gilson and Sripen Tantivess for their comments on an earlier version of this article. The workshop was jointly organized by the Overseas Development Institute (ODI), the HLSP Institute, the University of the Witswatersrand, the London School of Hygiene and Tropical Medicine, and King's College London, and funded by the British Academy and the UK Department for International Development. Comments by a peer-reviewer were particularly helpful in clarifying some of the arguments.
|
Endnotes |
|---|
|
TopAbstractIntroduction: the case for...Supporting advocacy coalitions...Doing research on the...DiscussionConclusionsEndnotesReferences |
|---|
1 Limited success is not surprising. Policy processes remain subject to chance and serendipity and, consequently, the most comprehensive and systematic attempts to collect and use information to influence policy processes may prove unpredictable and less successful than hoped for.
2 See Nelson (2004) which provides accounts of prospective analysis by national stakeholders. Some accounts are available but unpublished (due to the sensitivities of participants or the requirements of sponsors) or are published in the non-health literature, for example there is an unpublished account of DFID issue-based support to Nigeria (Pycroft and Butterworth 2005).
3 Reasons might include limited interest and skills or more likely the lack of access of researchers to advocacy groups with which they can collaborate on such research. It is also likely that because of institutional biases there is limited (and no commercial) sponsorship for the pro-poor prospective policy analysis sport.
4 Similar approaches could be used with other political actors. The rationale for the focusing on advocacy coalitions is set out below.
5 The issues raised by external participation in such processes are discussed below.
6 The benefits of an ‘outsider’ (which may or may not include an expatriate researcher) to the network undertaking research on the efficacy of the prospective analysis include greater objectivity, but insider/outsider issues are raised as discussed in Walt et al. (2008, this volume).
7 In this context, evidence refers to knowledge about the efficacy, effectiveness and efficiency of public health interventions as derived from scientific studies.
8 There may not be an evidence base for many interventions, and in many cases where evidence exists, there will be contestation, but this may be politically motivated and possible to see through. It may be useful for the policy analyst to discuss the appropriateness of the public health interventions with technical experts.
9 This is not to say that there is no capacity within pro-poor, health advocacy coalitions in low and middle income countries, but rather to acknowledge that there are limits to such capacity (see Gilson and Raphaely (2008, this issue).
10 In one of the few studies in the health sector which touches on this difficulty, De la Jara and Bossert explore the role of antecedents, interest groups and consensus building in the Chilean health reforms, and conclude that the processes are inherently complex and not captured by simple hypotheses (De la Jara and Bossert 1995).
11 In selecting such variables, Etzioni (2006) provides some valuable advice. In particular, he stresses the need to identify variables which are malleable, as he questions the sense of studying variables that we cannot change (e.g. sex or age composition of a population). He proposes instead to look at costs, acceptability to decision makers and the public. Social relations, including patterns of asset distribution and power, are of limited malleability whereas symbolic relations are highly malleable.
12 These challenges are not described in the literature but have been experienced by other policy analysts (personal communication) and the author.
|
References |
|---|
|
TopAbstractIntroduction: the case for...Supporting advocacy coalitions...Doing research on the...DiscussionConclusionsEndnotesReferences |
|---|
Bachrach P, Baratz M. Two faces of power. American Political Science Review (1962) 56:947–52.[CrossRef][Web of Science]
Bardach E. Policy dynamics. In: The Oxford handbook of public policy.—Moran M, Rein M, Goodin RE, eds. (2006) Oxford: Oxford University Press.
Bardach E. The skill factor in politics: repealing the mental commitment laws in California. (1972) Berkeley, CA: University of California Press.
Batley R. The politics of service delivery reform. Development and Change (2004) 35:31–56.[CrossRef][Web of Science]
Bossert T. Methodological guidelines – enhancing the political feasibility of health reform in Latin America. (2000) LAC/HSR 38. Boston, MA: Harvard School of Public Health.
Buse K, Mays N, Walt G. Making health policy. (2005) Milton Keynes, UK: Open University Press.
Buse K, Martin-Hilber A, Widyantoro N, Hawkes SJ. Management of the politics of evidence-based sexual and reproductive health policy. The Lancet (2006) 368:2101–03.
Campos JE, Syquia JL. Managing the politics of reform: overhauling the legal infrastructure of public procurement in the Philippines. In: World Bank Working Paper No. 70. (2006) Washington, DC: World Bank.
Cohen M, March JG, Olsen JP. A garbage can model for rational choice. Administrative Science Quarterly (1972) 1:1–25.
Collins C, Omar M, Tarin E. Decentralization, health care and policy process in the Punjab, Pakistan in the 1990s. International Journal of Health Planning and Management (2002) 17:123–46.[CrossRef][Web of Science][Medline]
de la Jara JJ, Bossert T. Chile's health sector reform: lessons from four reform periods. Health Policy (1995) 32:155–66.[CrossRef][Web of Science][Medline]
Downs A. Ups and downs with ecology: the issue-attention cycle. Public Interest (1972) 28:38–50.
Etzioni A. The unique methodology of policy research. In: The Oxford handbook of public policy.—Moran M, Rein M, Goodin RE, eds. (2006) Oxford: Oxford University Press.
Fischer F. Reframing public policy: discursive politics and deliberative practices. (2003) New York: Oxford University Press.
Galston WA. Political feasibility: interests and power. In: The Oxford handbook of public policy.—Moran M, Rein M, Goodin RE, eds. (2006) Oxford: Oxford University Press.
Gibson B. Beyond ‘two communities’. In: Evidence-based health policy: problems and possibilities.—Lin V, Gibson B, eds. (2003) Melbourne: Oxford University Press.
Giddens A. Central problems in social theory: action, structure and contradiction in social analysis. (1979) London: Macmillan.
Gilson L, Raphaely N. The terrain of health policy analysis in low and middle income countries: a review of published literature 1994–2007. Health Policy and Planning (2008) 23.
Gilson L, Doherty J, Lake S, et al. The SAZA study: implementing health financing reform in South Africa and Zambia. Health Policy and Panning (2003) 18:31–46.[CrossRef]
Glassman A, Reich MR, Laserson K, Rojas F. Political analysis of health reform in the Dominican Republic. Health Policy and Planning (1999) 14:115–26.
González-Rossetti A, Bossert T. Enhancing the political feasibility of health reform: a comparative analysis of Chile, Colombia and Mexico. In: Latin American and Caribbean Regional Health Sector Reform Initiative. (2000) Boston, MA: Data for Decision Making project.
Goodin RE, Rein M, Moran M. The public and its policies. In: The Oxford handbook of public policy.—Moran M, Rein M, Goodin RE, eds. (2006) Oxford: Oxford University Press.
Grindle MS. Despite the odds: the political economy of social sector reform in Latin America. In: Exclusion and engagement: social policy in Latin America.—Abel C, Lewis CM, eds. (2002) London: Institute for Latin American Studies.
Grindle MS, Thomas JW. Public choices and policy change: the political economy of reform in developing countries. (1991) Baltimore, MD: Johns Hopkins University Press.
Hogwood BW, Gunn LA. Policy analysis for the real world. (1984) New York: Oxford University Press.
John P. Analysing public policy. (1998) London: Cassell.
Immergut EM. Institutional constraints on policy. In: The Oxford handbook of public policy.—Moran M, Rein M, Goodin RE, eds. (2006) Oxford: Oxford University Press.
Kajula PW, Kintu F, Barugahare J, Neema S. Political analysis of rapid change in Uganda's health financing policy and consequences on service delivery for malaria control. International Journal of Health Panning and Management (2004) 19:S133–53.[CrossRef]
King G, Keohane RO, Verba S. Designing social inquiry: scientific inference in qualitative research. (1994) Princeton, NJ: Princeton University Press.
Kingdon J. Agendas, alternatives and public policies. (1984) Boston, MA: Little Brown and Co.
Klein R, Marmor TR. Reflections on policy analysis: putting it together again. In: The Oxford handbook of public policy.—Moran M, Rein M, Goodin RE, eds. (2006) Oxford: Oxford University Press.
Lavis JN, Robertson D, Woodside JM, et al. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Quarterly (2003) 81:221–48.[CrossRef][Web of Science][Medline]
Leichter H. A comparative approach to policy analysis: health care policy in four nations. (1979) Cambridge: Cambridge University Press.
Leighton C. Strategies for achieving health financing reform in Africa. World Development (1996) 24:1511–25.[CrossRef][Web of Science]
Lomas J, Culyer T, McCutcheon C, McAuley L, Law S. Conceptualizing and combining evidence for health system guidance. (2005) Ottawa: Canadian Health Services Research Foundation.
Lukes S. Power: A radical view. (1974) London: Macmillan.
Marsh D, Smith MJ. Understanding policy networks: towards a dialectical approach. Political Studies (2000) 48:4–21.[CrossRef][Web of Science]
Nelson JM. The politics of health sector reform: cross-national comparisons. In: Crucial needs, weak incentives. Social sector reform, democratization, and globalization in Latin America.—Kaufman RR, Nelson JM, eds. (2004) Washington, DC: Woodrow Wilson Center Press: co-published by The Johns Hopkins University Press.
Nelson JM. The politics of social sector reforms. (2000) Washington DC: Overseas Development Council. Online at: http://www.iadb.org/sds/doc/nelson1.pdf.
Nelson JM. Reforming health and education: the World Bank, the IDB and Complex Institutional Change. In: Overseas Development Council Policy Essay 26. (1999) Baltimore: Johns Hopkins University Press.
North DC. Institutions. Journal of Economic Perspectives (1991) 5:97–112.[Web of Science]
O’Brien B, Janeja M, Gopalakrishnan A. Applied interest-group analysis in reproductive health policy. The Lancet (2006) 369:368.
Oliver T. The politics of public health policy. Annual Review of Public Health (2006) 27:195–233.[CrossRef][Web of Science][Medline]
Olson M. The logic of collective action: public goods and the theory of groups. (1974) Cambridge, MA: Harvard University Press.
Page EC. The origins of policy. In: The Oxford handbook of public policy.—Moran M, Rein M, Goodin RE, eds. (2006) Oxford: Oxford University Press.
Parsons W. Public policy. (1995) Aldershot, UK: Edward Elgar.
Pycroft C, Butterworth R. Where capacity is not the only problem: Moving from generic capacity building to support for issues-based change in Nigeria (2005) Unpublished.
Rawls J. A theory of justice. (1971) Cambridge, MA: Harvard University Press.
Reich MR, Cooper DM. PolicyMaker: computer-assisted political analysis. In: Software, version 2.3.1. (1996) Brookline, MA: PoliMap. 1996–2001.
Reich MR. The politics of agenda setting in international health: child health versus adult health in developing countries. Journal of International Development (1995a) 7:489–502.[CrossRef][Medline]
Reich MR. The politics of health sector reform in developing countries: three cases of pharmaceutical policy. Health Policy (1995b) 32:47–77.[CrossRef][Web of Science][Medline]
Rhodes RAW. Policy network analysis. In: The Oxford handbook of public policy.—Moran M, Rein M, Goodin RE, eds. (2006) Oxford: Oxford University Press.
Roberts MJ, Hsiao W, Berman P, Reich MR. Getting health reform right: a guide to improving performance and equity. (2004) New York: Oxford University Press.
Sabatier PA, Jenkins-Smith HC. Policy change and learning: an advocacy coalition approach. (1993) Boulder, CO: Westview Press.
Schattschneider EE. The semisovereign people. (1966) New York: Holt: Rinehart and Winston.
Shapiro I. The flight from reality in the social sciences. (2005) Princeton, NJ: Princeton University Press.
Short S. Elective affinities: research and health policy development. In: Health policy in Australia.—Gardner H, ed. (1979) Melbourne: Oxford University Press.
Swank D. Global capital, political institutions, and policy change in developed welfare states. (2002) Cambridge: Cambridge University Press.
Thomas S, Gilson L. Actor management in the development of health financing reform: health insurance in South Africa, 1994–1999. Health Policy and Planning (2004) 19:279–91.
Thomas JW, Grindle MS. Political leadership and policy characteristics in population policy reform. Population and Development Review (1994) 20(Suppl.):51–70.[CrossRef][Web of Science]
Thompson GN, Estabrooks CA, Degner LG. Clarifying the concepts in knowledge transfer: a literature review. Journal of Advanced Nursing (2006) 53:691–701.[CrossRef][Web of Science][Medline]
UK Cabinet Office. Professional policy making for the 21st Century. (1999) London: Cabinet Office.
van de Walle N. African economies and the politics of permanent crisis, 1979–1999. (2001) Princeton, NJ: Princeton University Press.
Walt G, Shiffman J, Schneider H, et al. ‘Doing’ health policy analysis: methodological and conceptual reflections and challenges. Health Policy and Planning (2008) 23.
Walt G. Health policy: an introduction to process and power. (1994) London: Zed Books.
Walt G, Gilson L. Reforming the health sector in developing countries: the central role of policy analysis. Health Policy and Planning (1994) 9:353–70.
Weiss CH. The many meanings of research utilization. Public Administration Review (1979) 39:426–31.[CrossRef][Web of Science]
Accepted for publication 22 June 2008.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  K Buse, N Lalji, S H Mayhew, M Imran, and S J Hawkes Political feasibility of scaling-up five evidence-informed HIV interventions in Pakistan: a policy analysis Sex Transm Inf, April 1, 2009; 85(Suppl_2): ii37 - ii42. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K. Buse, C. Dickinson, and M. Sidibe HIV: know your epidemic, act on its politics J R Soc Med, December 1, 2008; 101(12): 572 - 573. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||