Health Policy and Planning Advance Access published online on March 30, 2007
Health Policy and Planning, doi:10.1093/heapol/czm008
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
A rapid assessment methodology for the evaluation of primary care organization and performance in Brazil
1Robert Wood Johnson Foundation Health and Society Scholars Program, University of Pennsylvania, Philadelphia, PA, USA.
2Department of Health Administration and Planning, National School of Public Health/Oswaldo Cruz Foundation, Rio de Janeiro, Brazil.
3Faculty of Medicine of Petrópolis, Municipal Health Secretariat, Petrópolis, Brazil.
*Corresponding author. RWJF Health & Society Scholars Program, University of Pennsylvania, 3641 Locust Walk, 309 Colonial Penn Center, Philadelphia, PA 19104-6218, USA. Tel: +1 215 746-3171; Fax: +1 215 746-0397; E-mail: macinko{at}wharton.upenn.edu
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionConclusionsAppendix AReferences |
|---|
Introduction This study presents a methodology for the rapid assessment of the organization and performance of primary care services at district level. It compares results from an earlier provider survey in Petrópolis, Brazil with those obtained directly from users of two types of primary care services (newly developed Family Health Program services or ‘PSF’—Programa de Saúde da Família—and traditional services) in the same municipality. The aim is to demonstrate the validity of the methodology and its potential use in measuring and improving district-level primary care services in developing countries.
Methods The study adapted a previously developed questionnaire to measure essential dimensions of primary care. Users (n = 468) were randomly selected from each of the 40 primary care clinics in the district and administered a question survey. Responses were used to create nine measures: an overall composite ‘total primary care index’ and eight sub-indices each pertaining to an essential primary care dimension.
Results Primary care services show considerable variation in user experiences. Users of the new (PSF) clinics reported higher overall assessments of the total primary care index and the sub-indices for gatekeeping, comprehensiveness, family focus and community orientation than did users of traditional services (P < 0.05). The total primary care score was internally consistent with a Cronbach's alpha of 0.8, and could be reduced to only one principal component. User assessments of primary care services were predicted by self-rated health (OR 1.72) and the site of care (OR 1.03). User and provider assessments of the total primary care index were not significantly different, disagreeing on only two sub-indices (gatekeeping and family focus, P < 0.05).
Conclusions The study presents a rapid and valid method of obtaining information about clinic-level variation in primary care organization and performance at district level. The total primary care index was not sensitive to demographic or socio-economic characteristics of clients, was internally consistent and appears valid given similar results obtained using two different data sources. With some adaptation the instrument could potentially be applied in other developing countries.
Key Words: Health care reform, primary health care, Brazil, health services evaluation
KEY MESSAGES
|
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionsAppendix AReferences |
|---|
Primary care represents the basis for many health systems by providing accessible and comprehensive preventive and curative care for all of the most common population health needs and by coordinating care with other levels when referral is necessary (Starfield 1998
). However, primary care can be the ‘weak link in the chain’ in health systems due to health sector reforms and other processes that have left front-line services with inadequate resources, staffing and management capacity (Freedman et al. 2005). This study presents results from Brazil, which over the past 15 years has invested considerably in the reform of its health system, including the development of new models of organizing and delivering primary care. In this article, we use the term ‘primary care’ to refer to the first level of care provided in the health system and ‘clinics’ to refer to government health posts or centres that provide primary care.
Brazil has undergone considerable health sector reforms since the 1980s. The national health policy is based on the Federal Constitution of 1988, which institutionalized universal social rights, — including the right to health, which then became a citizen's right and the State's obligation and responsibility, — and set out the basis of the Unified Health System (the Sistema Único de Saúde or SUS) (Almeida et al. 2000). Principles of the SUS are equitable access to health care and universality (Almeida 2000).
Most Brazilians receive primary care through public facilities (health posts and centres) and since the mid-1990s several strategies have been pursued to improve the availability and effectiveness of primary care services as a priority of the national health policy. These include the development of special programmes such as the Family Health Program (known as the Programa Saúde da Família or PSF) and the Program of Community Health Agents (Programa de Agentes Comunitários de Saúde or PACS). The PSF includes a health care team composed of a physician, a nurse and community health agents who regularly visit each household in a given geographic district (Ministry of Health of Brazil 2001; Ministry of Health of Brazil 2005). While originally begun as a programme for impoverished areas, objectives of the PSF now include reorganizing primary care to become the gatekeeper of the health system, integrating primary care with other health and social services, changing the focus of care from the individual to the family, and improving community participation and outreach. The programme is funded by per capita financing known as the Primary Care Grant (Piso de Atenção Básica or PAB) and additional financial incentives are provided to municipalities for reaching higher levels of population coverage (Ministry of Health of Brazil 2002).
In 2006, the PSF covered about 85 million people or nearly 45% of the Brazilian population, making it one of the largest systems of community-based primary care in the developing world. Even though the PSF is now considered the principal strategy for reorienting the Brazilian health system toward primary care, there are currently no standard tools available to assess primary care performance at the local level.
The project described here was carried out in close collaboration with municipal health authorities. It responds to their need for a rapid and valid way of measuring changes in primary care organization and service delivery. The pilot stage of this project employed health providers and managers as key informants. During the pilot stage (conducted in June 2003), a questionnaire was developed and applied to health providers (nurses or physicians) and district supervisors in the city of Petrópolis. Results of the pilot survey supported hypotheses that the new (PSF) services performed better on most primary care dimensions. However, results showed that the traditional primary care services appeared, in some cases, to offer a more comprehensive range of services and were somewhat more accessible (Macinko et al. 2004). Despite the interest in using providers, managers and supervisors as key informants to rapidly assess clinic-level performance, the question was raised as to the validity of key informants’ opinions. That is, were provider opinions correlated with the actual experiences of clinic users? The purpose of the current study is to present data on the validation of this methodology and to provide an illustration of how managers and health care providers might use these methods to improve decision-making in the context of a continuously evolving health system.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionsAppendix AReferences |
|---|
Population and sampling methodology
The setting for the study is the municipality of Petrópolis located about 2 hours outside the city of Rio de Janeiro. Petrópolis was chosen because of its relatively large and diverse population, its role as an early adopter of the PSF programme, the receptiveness of the local health secretariat for evaluation activities, and the municipality's tradition of investments in primary care.
The study universe consists of all adult users of government primary care facilities in the municipality of Petrópolis (total population of approximately 290 000). Government services provide nearly all primary care for the poorest 75% of the population that does not have private health insurance (PAHO 2002). Every public primary care site was included in the survey, including all 25 PSF clinics and all 15 traditional services (which include 11 health posts, two health centres, one ambulatory clinic within the main hospital and one primary care clinic within an urgent care centre).
The primary sampling unit (PSU) is the primary care clinic, since a neighbourhood sample is complicated by the large territory and mountainous terrain of Petrópolis. Using primary care clinics as the PSU has the additional advantage of capturing users who have just experienced primary care, thereby minimizing recall bias and linking each user with the place in which he or she received care.
Sample size was calculated using the following formula:
|
|
/2) = value of the normal distribution corresponding to the probability of a type 1 error of 0.05; Z(1–ß) = value of the normal distribution corresponding to a probability of a type 2 error of 0.8; 
= the difference in the mean value of the primary care score between the traditional and PSF clinics; 
= standard error of these means. To control for clustering by clinic, we multiply this formula by the design effect: 1 + 
(m – 1), where = intra-class correlation and m = the number of observations per cluster (Levy and Lemeshow 2003). We then used Stata commands sampsi and sampclus to generate a range of possible values (StataCorp 2003). A target sample size was set to 480 people—or 12 per clinic—given type 1 error of 0.05, type 2 error at 0.8, a maximum mean difference in scores of 0.5, and intra-class correlations of up to 0.3. The final sample size was 468 due to a 2.5% refusal or incompletion rate, even though each clinic had a minimum of 11 interviews completed. Based on these results, the average design effect was 3.21 (average intraclass correlation of 0.28) meaning the study was adequately powered given its final sample size. Users were randomly selected at each clinic (during two consecutive days) from among those with an appointment between January and February 2004. Prior consultation with health care providers ensured that no interviews occurred on holidays or on days that might be otherwise considered unusual. All interviewers were hired from a local Brazilian research centre, had at least an undergraduate degree and at least 1 year prior experience working as an interviewer. Interviewers were trained on the instrument and its application, were involved in the pre-test and had daily supervision by a senior researcher.
Instrument
The study builds upon the Primary Care Assessment Tool (PCAT) that has previously been used to evaluate user and provider assessments of primary care services in the United States and elsewhere. The instrument has been found to be a valid and reliable method of measuring primary care experiences for adults and children (Cassady et al. 2000; Shi et al. 2001). The PCAT measures the main dimensions of primary care: access, first-contact (or gatekeeping), longitudinality, comprehensiveness, coordination, family focus and community orientation (see Appendix for a description of each dimension). These dimensions have been found to be among the most salient in defining effective primary care service organization and delivery at the population level (Starfield and Shi 2002; Macinko et al. 2003). In this study, we translated the PCAT into Portuguese and adapted it by changing some items to reflect Brazilian realities (modifying the list of services offered in primary care to meet Brazilian guidelines, adding items to measures of community and family orientation, for example) and including an additional scale assessing the skills and training of primary care providers.
Like the PCAT instrument, the questionnaire used in this study employs a core set of about 100 questions intended to measure each primary care dimension. As in the PCAT, questions do not differ significantly from the provider or user versions of the questionnaire. For example, provider questions are in the form of ‘In this clinic, how often can most clients obtain X?’, while client questions would be formed as, ‘In your visits to this clinic, how often could you obtain X?’ The user survey also included questions about self-assessed health status and other demographic information. The final instrument was pre-tested with users of a primary care clinic in Rio de Janeiro before its use in Petrópolis.
Users provided their assessments of specific primary care experiences by responding to each question using a six-point Likert-type scale ranging from 0 (never) to 5 (always). Respondents were shown a response card that graphically demonstrated the different response categories and the distance between them. The questionnaire asks users to evaluate specific aspects of the receipt of primary care services rather than soliciting opinions about their levels of ‘satisfaction’ since the latter term has been shown to be difficult to quantify (Schneider and Palmer 2002).
Data analysis
In this study, all responses are dichotomized (0, 1, 2 are coded as ‘0’ and 3, 4, 5 are coded as ‘1’). This was done to minimize any potential bias introduced by respondent difficulty in effectively discriminating between some responses (between ‘almost never’ and ‘sometimes’, for example), a common technique in survey research (McDowell 2006). In sensitivity tests, using the original scaling (i.e. 0–5) had a minimal effect on the main conclusions presented here.
Eight indices (or scales) were created from the survey questions, one for each primary care dimension (e.g. access, comprehensiveness, etc.). This was accomplished by summing all responses to create an average score for that dimension. A ‘total primary care score’ was also created by taking the average of all eight dimension-specific scales.
Data analyses included description of the population sampled, distribution of each primary care index and comparison of results obtained by type of health facility (e.g. PSF or traditional).
Psychometric properties of the instrument were assessed by calculating Cronbach's alpha as a measure of internal consistency reliability, and by employing principal components factor analysis to assess whether the total primary care score was, in fact, measuring a single latent construct.
We employed multiple logistic regression to examine determinants of users’ primary care scores, such as demographic and health status characteristics. Because the total primary care score cannot be considered a purely continuous variable (it is bounded by 0 and 1), we constructed a binary dependent variable to represent scores above or below the median score. Independent variables included self-assessed health status (coded as ‘1’ if the user responded their health was ‘good’ or ‘very good’, and ‘0’ if they responded ‘average’, ‘poor’ or ‘very poor’); chronic disease (‘1’ if the user reported having any of the following illnesses: diabetes, hypertension, cancer, arthritis, asthma, heart disease or depression); recent illness (‘1’ if the user reported being unable to perform any regular activities due to illnesses anytime during the past month); and household wealth (a scale that varied from 0 to 8) that consisted of the number of household goods the user reported having at home: electricity, piped water, bathroom, refrigerator, radio, telephone, television and car.
A final test of the instrument's validity involved comparing the association between provider and user assessments of the same primary care dimensions in the same clinic.
For every analysis, standard errors were adjusted for clustering by clinic. All analyses were performed using Stata (StataCorp 2003).
The study included only adults, defined as individuals aged 18 or older. Free and informed consent was obtained from all participants. This research was approved by the Committee on Ethics in Research of the National School of Public Health Sergio Arouca/Oswaldo Cruz Foundation, Rio de Janeiro, Brazil, and the New York University Committee on Human Subjects Research.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionsAppendix AReferences |
|---|
Table 1 presents demographic and health data on the users interviewed. They were predominantly female, with an average age of 40 years, and with a generally low level of education (75% did not complete primary school). Almost everyone had basic household needs met (electricity, piped water, indoor bathroom, television), but there are indications that they are nevertheless of lower socio-economic status. For example, only 68% had their own telephone and about a quarter had a car. The majority reported good or excellent health status, but nearly 60% reported at least one chronic condition. Forty per cent reported having hypertension and 20% reported depression. About a third of all users had more than one chronic condition. There were no significant differences between users of PSF and users of traditional services in terms of health status, demographic or economic characteristics.
|
Table 2 compares primary care indices, by dimension, for PSF and traditional clinics. The dimensions of gatekeeping, comprehensiveness, family and community orientation were significantly higher for PSF users than for users of traditional services (P < 0.05). The PSF users reported about a 20% higher overall evaluation of their primary care experience than those in the traditional primary care services (P < 0.05). Both types of clinics scored poorly in terms of access.
|
Figure 1 shows the combined user assessments of primary care experience by clinic and service type. The graph shows that there was considerable variation in primary care indexes within clinics and between clinic types. Of particular note are PSF clinics 16 and 17, with scores significantly higher than any other clinic. In addition, five traditional clinics had scores equal to or higher than the lowest median scores obtained by PSF clinics.
|
Table 3 presents results of the Cronbach's alpha test of internal consistency reliability. The ‘sign’ column shows that all variables are scaled in the same direction. Item-test correlations show that each index is strongly associated with the total primary care scale, with values ranging from 0.61 for access to 0.71 for providers. The item-rest column presents correlation coefficients for the item and the scale that is formed by all of the other items. These values range from 0.47 for access to 0.60 for providers. The final column presents the value of Cronbach's alpha that would result by dropping the item in question from the total primary care score. The overall Cronbach's alpha for the total primary care score (including all eight items) was 0.80.
|
Results (data not shown) of a principal components factor analysis of the total primary care score composed of all eight primary care sub-scales indicate that the eight items could be reduced to only one factor with an Eigen value over 1. The next highest Eigen value (for a second potential factor) was less than one and occurred at the inflection point in the scree plot.
Table 4 presents results of a logistic regression analysis designed to assess determinants of an individual's total primary care score. The variables age, gender, household wealth, presence of at least one chronic health condition, and the site of care (PSF vs. traditional clinics) did not significantly predict total primary care score. Significant variables include self-perceived health status (those in better health are 1.7 times more likely to report higher scores than those in poorer health) and the specific clinic where treatment was received.
|
Table 5 compares results obtained from users with those obtained from providers from the previous stage of the research. Evaluations of primary care performance between users and health care providers were similar in all cases except assessments of gatekeeping (users evaluated it more highly than did providers) and family focus (users’ rating was more negative than the providers’). The same pattern was found for PSF and traditional clinics. In neither case were there significant differences between users and providers for the total primary care score.
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionsAppendix AReferences |
|---|
The PSF was explicitly developed in order to better attain the dimensions of primary care discussed here. In the case of Petrópolis, it seems to have succeeded in improving on traditional services in six out of the eight dimensions of primary care. This result was consistent regardless of the data source (users or key informants). However, even though average total primary care scores were higher in PSF clinics, variations in scores within each programme were considerable. For example, several of the highest performing traditional primary care clinics had scores equal to or higher than the lowest performing PSF clinics. This suggests that greater attention could be paid to the process of care and indicates the possibility for quality improvement initiatives to learn how the best rated clinics achieved such a high level of performance, and to transfer lessons learned to other clinics.
The main areas in which the PSF outperformed traditional services included gatekeeping, comprehensiveness, family focus and community orientation. These dimensions (at least as measured in this study) represent aspects of primary care organization and delivery that depend mostly on the day-to-day operations of each clinic. Areas where the two clinic types did not differ (such as access, coordination and provider skills) are more dependent on the local infrastructure (such as availability of pharmaceuticals), overall physician supply and training, and municipal-level rules governing the official working hours of government services and personnel. Based on the results presented here, it appears that improving primary care in Petrópolis is likely to require both clinic-level and municipal-level changes.
These descriptive results are supported by the results of tests of the validity and reliability of the survey instrument. These results suggest that the total primary care score meets basic standards for scale reliability and validity. The results for Cronbach's alpha show that no individual sub-scale's values were higher than that for the total primary care score, suggesting that all of the items are relevant to the total scale. The value of Cronbach's alpha for the total primary care score was 0.80, suggesting that it achieved a good level of internal consistency reliability (DeVellis 1991).
Principal component factor analysis provides additional evidence of the scale's validity. Criteria for selecting the number of factors to retain after principal components analyses include using factors with Eigen values over 1, and only using factors that occur above the inflection point in the scree plot (Bartholomew et al. 2002). Both criteria support the hypothesis that the total primary care score presented here measures one single factor or latent construct.
Multivariable analyses of individual-level determinants of the primary care score did not vary by demographic or socio-economic characteristics of the respondents, further bolstering the evidence that the questionnaire was able to accurately measure users’ primary care experience independent of potential confounders such as age, gender, poverty or educational levels.
The significance of the positive association between self-reported health status and higher primary care scores is less clear. This relationship could be due to the fact that people who are in better health might be more likely to rate a service of better quality. However, in multiple regression analyses the relationship held even while controlling for other health status indicators such as chronic disease status and recent illness. Moreover, a recent cross-sectional study showed that the national prevalence of good/excellent self-reported health was about 53%, that one-third of the population reported some kind of long-term illness, and that about 20% suffered from depression (Szwarcwald et al. 2005), figures quite similar to the health profile of the users surveyed in Petrópolis. This suggests that the population surveyed was not significantly more healthy (or more ill) than would be expected in a population or household survey. The positive association between self-rated health and better primary care experiences also held in analyses stratified by poverty, perceptions of ease of access, and by chronic disease state (results not shown), further decreasing the likelihood that the primary care scale is somehow confounded by sub-group characteristics. One alternative explanation is that those who report better primary care experiences might actually have improved health as a result of that care. Unfortunately, the cross-sectional nature of this study makes it impossible to ascertain the true direction of this relationship.
Finally, comparisons between user and provider experiences of primary care also yielded several insights. There was no significant difference in provider or user evaluations of six out of the eight sub-indexes, and there was no difference in evaluations of the total primary care index. One implication of this finding is that the provider survey could be used to calculate the total primary care score to monitor general aspects of primary care performance at the municipal level. For clinics with particularly low or high scores, user surveys could then be implemented to offer additional information on specific primary care dimensions, to assess variation among clinic types, or to assess variation among different types of clients. Ultimately, the right mix of strategies will depend on the level of detail required and the resources available to managers at the district level.
Limitations
This study has some limitations. First, it did not collect data on the technical quality of primary care as it was delivered. Thus, we cannot directly test whether better organization and delivery of primary care is associated with improved quality of care received. A further validation exercise would include assessing how consistently clinically appropriate care is delivered within each clinic and then measuring how closely this is correlated with user and provider assessments of primary care organization and performance.
Secondly, the study results are not necessarily generalizable to municipalities outside Petrópolis. That is, we cannot say that the relationship between user evaluations of PSF and traditional clinics would be the same in another location within Brazil. Nevertheless, the purpose of this study was to illustrate how the proposed methodology could be used at the district level. Future studies should replicate the approach used here in more diverse environments by adapting the instrument and study design to meet local needs.
Thirdly, this study was not designed to analyse the location or spatial distribution of PSF or traditional services. Equity is an expressed value of the Brazilian health system, and additional analyses should be carried out to ascertain whether the expansion of PSF clinics and the variations in clinic performance are associated with inequalities in access, use and health outcomes among different population groups.
Finally, because of resource limitations, this study did not include non-users of primary care (i.e. those who would be captured in a household rather than a clinic-based survey). Their experiences could be different from regular users of government clinics in some systematic way. Including non-users could shed light on additional determinants of access and use of government services. For example, it could be that all users who had poor opinions about the quality of primary care no longer seek care at government sites and might instead turn to the private sector. In Petrópolis (as in most of Brazil), this phenomenon is likely to be minimized since private sector health providers are primarily used by individuals with private or supplemental health insurance that is generally only affordable to people with much higher income than those surveyed here (Viacava et al. 2005). This study did include other government options such as health centres, emergency clinics and hospital ambulatories, reflecting the full range of government primary care services, none of which require health insurance or payment for services. Nevertheless, future studies should compare non-users of public clinics with current users in order to adequately control for any potential selection bias. Comparing regular users with infrequent users or those who use only the private sector would be especially important in most other developing countries where there is no universal health system.
|
Conclusions |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionsAppendix AReferences |
|---|
This study has presented results on the validity and potential uses of a method for measuring primary care organization and performance at the district level. Research in the United States and in European countries has suggested that primary care services that achieve the dimensions of primary care assessed here are associated with lower costs, better quality of care and improved outcomes (Starfield 1994
; Elola et al. 1995; Macinko et al. 2003). But there is surprisingly little evidence of the impact of the full range of primary care services (as opposed to specific health interventions such as oral rehydration therapy or immunizations) on population health in developing countries (Hill et al. 2000). This absence may be partly explained by the lack of available, valid tools. It is hoped that the instrument and methodology used in this study could help to fill this gap. In order to aid this process, the Portuguese-language instruments and a computer program to aid in data entry and analysis are available for download at no cost through the Pan American Health Organization's office in Brazil (at http://www.opas.org.br/servico/temas_documento_detalhe.cfm?CodSubTema=178&CodEspecifico=5564).
Our experience in using this methodology suggests it could be used for periodic monitoring and evaluation of primary care at the district level. Each questionnaire took less than 20 minutes to administer and each clinic could interview a sample of clients in 1–2 days’ time—less time than is currently spent on performance data collection, which, according to a study conducted in 10 large Brazilian urban centres, managers did not consider useful and therefore did not use in regular planning and evaluation activities (Escorel et al. 2002). Discussions with municipal managers in Petrópolis revealed that the results of the provider and user surveys were easily understood and, perhaps more importantly, readily linked to specific actions to improve the organization and delivery of care at both the clinic and district levels.
Ultimately, it is impossible to say that the mere availability of validated tools for the rapid assessment of primary care will make any difference to performance without the development of incentives linked to improvements in processes and outcomes. In Brazil, financial incentives for municipalities are currently linked to increasing population coverage, with few systematic monitoring and evaluation processes in place. A major challenge of the PSF programme as it moves forward will be to develop and use systems to monitor and improve the quality of care delivered in order to maximize the potential health gains of this innovative approach to integrated primary care delivery.
|
Appendix A |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionsAppendix AReferences |
|---|
Survey questions for each primary care indexa,b
|
a All responses are coded as follows: 0 = never; 1 = almost never; 2 = sometimes; 3 = many times; 4 = almost always; 5 = always.
b This English-language translation has not been field-tested.
Source: Adapted from the Primary Care Assessment Tool (Starfield 1998).
|
Acknowledgements |
|---|
This study was funded by the Strategic Research Program (PED) of the National School of Public Health Sergio Arouca/Oswaldo Cruz Foundation in Rio de Janeiro, the Pan American Health Organization Representation in Brazil, and the New York University. Additional support was provided by the Office of the Municipal Secretariat of Health of Petrópolis, Brazil.
Thanks to Barbara Starfield, Julio Suarez, Renato Möller, Eliane dos Santos Oliveira and Lisa Kroin.
The views expressed here are those of the authors’ alone.
Conflict of interest: None declared.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionsAppendix AReferences |
|---|
Almeida C. 2000. Saúde e equidade nas reformas contemporâneas. Saúde em Debate 24:6–21.
Almeida C, Travassos C, Porto S, Labra ME. 2000. Health sector reform in Brazil: a case study of inequity. International Journal of Health Services 30:129–62.[CrossRef][Web of Science][Medline]
Bartholomew D, Steele F, Moustaki I, Galbraith JI. 2002. The analysis and interpretation of multivariate data for social scientists.Boca Raton, FL Chapman & Hall.
Cassady CE, Starfield B, Hurtado MP, et al. 2000. Measuring consumer experiences with primary care. Pediatrics 105:4 Pt 2, 998–1003.
DeVellis R. 1991. Scale development: theory and applications.Newbury Park, CA Sage Publications.
Elola J, Daponte A, Navarro V. 1995. Health indicators and the organization of health care systems in Western Europe. American Journal of Public Health 85:1397–401.
Escorel S, Giovanella L, Mendonça MH, et al. 2002. Avaliação da implementação do Programa Saúde da Família em dez grande centros urbanos. Síntese dos principais resultados.Brasilia Ministério da Saúde, Departamento de Atenção Básica.
Freedman L, Wirth M, Waldman R, et al. 2005. Who's got the power? Transforming health systems to improve the lives of women and children. Millennium Project Task Force 4: Child Health and Maternal Health Final Report.New York UN Millennium Project.
Hill AG, MacLeod WB, Joof D, Gomez P, Walraven G. 2000. Decline of mortality in children in rural Gambia: the influence of village-level primary health care. Tropical Medicine and International Health 5:107–18.[CrossRef]
Levy P and Lemeshow S. 2003. Sampling of populations: methods and applications. 3rd edn New York John Wiley & Sons, Inc.
Macinko J, Almeida C, Oliveira ES, de Sa PK. 2004. Organization and delivery of primary health care services in Petropolis, Brazil. International Journal of Health Planning and Management 19:303–17.[CrossRef][Web of Science][Medline]
Macinko J, Starfield B, Shi L. 2003. The contribution of primary care systems to health outcomes within OECD countries, 1970–1998. Health Services Research 38:831–65.[CrossRef][Web of Science][Medline]
McDowell I. 2006. Measuring health: a guide to rating scales and questionnaires.New York Oxford University Press.
Ministry of Health of Brazil. 2001. Programa Saúde da Família [Family Health Program].Brasília Ministry of Health.
Ministry of Health of Brazil. 2002. Analise de reestruturação dos modelos assistenciais de saúde em grandes cidades: padrões de custos e forma de financiamento.Brasília Ministry of Health.
Ministry of Health of Brazil. 2005. Sistema de Informação de Atenção Básica (SIAB) [Primary Care Information System]. Online at: http://www.datasus.gov.br/siab/siab.htm, accessed 23 May 2005.
PAHO. 2002. Health in the Americas (Volumes I and II)Washington, DC Pan American Health Organization.
Schneider H and Palmer N. 2002. Getting to the truth? Researching user views of primary health care. Health Policy and Planning. 17:32–41.
Shi L, Starfield B, Xu J. 2001. Validating the adult primary care assessment tool. Journal of Family Practice 50:E1.[Medline]
Starfield B. 1994. Is primary care essential? The Lancet 344:1129–33.
Starfield B. 1998. Primary care: balancing health needs, services and technology.New York Oxford University Press.
Starfield B and Shi L. 2002. Policy relevant determinants of health: an international perspective. Health Policy 60:201–16.[CrossRef][Web of Science][Medline]
StataCorp. 2003. Stata user's guide: Release 8.College Station, TX Stata Corporation.
Szwarcwald CL, Souza-Junior PR, Esteves MA, Damacena GN, Viacava F. 2005. Socio-demographic determinants of self-rated health in Brazil. Cadernos de Saúde Pública 21:Suppl., 54–64.[Medline]
Viacava F, Souza-Junior PR, Szwarcwald CL. 2005. Coverage of the Brazilian population 18 years and older by private health plans: an analysis of data from the World Health Survey. Cadernos de Saúde Pública 21:Suppl., 119–28.[Medline]
Accepted for publication 25 January 2007.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||