Participatory Approaches for Study Design and Analysis in Dissemination and Implementation Research
Participatory Approaches for Study Design and Analysis in Dissemination and Implementation Research
Abstract and Keywords
This chapter seeks to demonstrate the value added from community participation to the research process itself. It shows how Community-Based Participatory Research (CBPR) methods are useful in the dissemination and implementation (D&I) of research findings and some of the lessons from CBPR for D&I research. It discusses challenges that can play out in participatory research, followed by a more detailed examination of the specific ways in which a CBPR approach can enhance the D&I of research findings through collaborative design, analysis, dissemination, and research translation. A case study of a community-university-health department CBPR project that endeavored to study and improve the health and working conditions of restaurant workers in San Francisco's Chinatown District is presented. Then, some of the methods used to involve all partners in study design, data analysis, and translation of findings into action, as well as some of the benefits of doing so, are discussed. Finally, key lessons learned, through this and other CBPR efforts, are shared, and their implications for improving the breadth and effectiveness of the critical dissemination and implementation phases of research are summarized.
The importance of community and other stakeholder participation for improving the quality and relevance of research has long been acknowledged.1 – 4 With the growing interest in closing the “chasm”5 between research and practice and more effectively eliminating health disparities, the potential benefits of participatory approaches for dissemination and implementation of research findings are increasingly being be considered. In particular, Community-Based Participatory Research (CBPR), with its commitment to action as part of the research process, holds great potential not only for improving the relevance of research to communities and stakeholders but also for ensuring that research results are effectively disseminated and translated into programs and policies to promote health. This partnership orientation to research has become recognized as an important means through which the distance between research and action might be more effectively bridged.6 Indeed, as Horowitz and her colleagues point out, “CBPR may be the ultimate form of translational research . . . moving discoveries bidirectionally from bench to bedside to el barrio (the community) to organizations and policy makers.”7 Although CBPR is not possible or applicable in all research contexts, when it is appropriate, much value can be added to the research process and subsequent dissemination and implementation of findings through high-level community and other stakeholder participation. As discussed below, the level of community engagement in research takes place along a continuum, from limited consultation on specific aspects of the study, through high-level co-collaboration at every step in the process.
For readers unfamiliar with CBPR, this chapter seeks to demonstrate the value added from community participation to the research process itself. It also shows how CBPR methods are useful in the dissemination and implementation (D&I) of research findings and some of the lessons from CBPR for D&I research. There is a large and growing literature on CBPR that cannot be covered in a single chapter, so entry points into this body of writing are provided. After briefly describing a continuum of such participation, the discussion is focused, in particular, on CBPR. Challenges that can play out in participatory research are discussed, followed by (p.193) a more detailed examination of the specific ways in which a CBPR approach can enhance the D&I of research findings through collaborative design, analysis, dissemination, and research translation. A case study of a community-university-health department CBPR project, in which both authors were involved, that endeavored to study and improve the health and working conditions of restaurant workers in San Francisco's Chinatown District is presented. Then, some of the methods used to involve all partners in study design, data analysis, and translation of findings into action, as well as some of the benefits of doing so, are discussed. Finally, key lessons learned, through this and other CBPR efforts, are shared, and their implications for improving the breadth and effectiveness of the critical dissemination and implementation phases of research are summarized.
Continuum of Participation in Research
Community and other stakeholder participation in research can be seen as occurring along a continuum, with benefits accruing at each stage, and the most substantial value added often occurring at the farthest end of the continuum. Over two decades ago, Biggs developed a continuum of community involvement in research, which remains frequently cited in the field.8 He described the levels of participation as (1) contractual, in which community and other stakeholders simply take part in researchers’ studies; (2) consultative, in which they are asked their opinions as interventions and other research instruments are designed and implemented; (3) collaborative, in which researchers and community and other stakeholders work together on projects designed, initiated, and managed by researchers; and (4) collegiate, in which all partners work as colleagues with different skills to offer, and there is mutual learning, and local control over the research process.
The second, third, and fourth levels in Biggs's model each can be seen as adding successively greater potential value to research processes and outcomes. Consultation with community and other stakeholders, for example, can result in higher recruitment rates and lower attrition, as well as fewer cultural and linguistic barriers, which in turn can increase the accuracy of data reporting and the appropriateness and effectiveness of interventions.9 – 11 Investigator-initiated and managed studies that emphasize true collaboration with community and other stakeholders can increase still further the relevance and efficacy of research, resulting in greater community buy-in and trust, interventions that are better tailored to the study population, and enhanced efficacy in data collection, interpretation, and dissemination.11 Finally, research that is truly collaborative in nature ensures that the research topic itself comes from, or is of substantial importance to, the community, and that the research process includes high levels participation throughout, including dissemination and use of study findings to help address the problem under study.3 , 11 , 12
Community-Based Participatory Research (CBPR) involves the most intensive form of community and stakeholder participation and typically takes place at the collaborative level. In this chapter, CBPR is used as an umbrella term for an orientation to research that goes by many names (among them participatory action research, community-partnered research, mutual inquiry and participatory research) and that (p.194) have in common a commitment to combining research, participation, education, and action.2 , 12 When feasible and appropriate, CBPR may be particularly well suited to translational research aimed at studying and addressing health disparities. This orientation to research adds value at each stage of the research process—from identification of the problem and of community assets to study design, analysis, and the dissemination and use of research findings. Although it also presents substantial challenges, including the time- and labor-intensive nature of the work, this orientation to research has achieved growing attention both in North America and internationally.
CBPR Definition and Principles
Community-Based Participatory Research is concisely defined by Green and his colleagues (1994) as “systematic inquiry, with the participation of those affected by the issue being studied, for the purposes of education and taking action or effecting change.”1 Building on this definition, as well as earlier work by Israel et al.,3 the Kellogg Community Health Scholars Program13 crafted a definition that situates CBPR within the context of efforts to study and address health disparities.3 , 13 In its words, CBPR is “A collaborative process that equitably involves all partners in the research process and recognizes the unique strengths that each brings. CBPR begins with a research topic of importance to the community with the aim of combining knowledge and action for social change to improve community health and eliminate health disparities.”13
The research dimension of CBPR can involve a wide range of qualitative and quantitative methods. Developing and administering community surveys or focus groups; conducting walkability assessments or air monitoring; using Geographic Information Systems (GIS) mapping; conducting secondary data analysis; and using randomized controlled trials (RCTs) to assess intervention effectiveness all have been used by as part of CBPR efforts.10 , 11 , 14 Regardless of the particular research methods used, however, what is unique about this orientation to research is the way in which the research is conceptualized and conducted,4 the heavy accent placed on genuine community and stakeholder engagement throughout the process, and the use of findings to help bring about change.
Eleven principles of CBPR (Table 10–1), described below, help to further articulate how CBPR differs from more traditional “top-down” approaches to research and is consistent with translational research that is indeed “community based,” rather than merely “community placed.” Israel and her community and academic colleagues3 , 15 developed nine guiding principles of CBPR that are widely used to inform and guide the process of CBPR. Two other principles, added subsequently by Minkler and Wallerstein,12 are also critical to this work. Although translational research partnerships wishing to utilize CBPR should adapt these principles (or develop new ones), as appropriate, given their own unique contexts, these eleven principles may be helpful in providing initial guidance. (p.195)
Table 10–1. CBPR Principles
1. Recognizes community as a unit of identity, whether community is defined in geographic, racial/ethnic or other terms.
2. Builds on strengths and resources within the community
3. Facilitates a collaborative, equitable partnership in all phases of research, involving an empowering and power-sharing process that attends to social inequalities.
4. Fosters colearning and capacity-building among all partners
5. Integrates and achieves a balance between knowledge generation and intervention for the mutual benefit of all partners.
6. Focuses on the local relevance of public health problems and on ecological perspectives that attend to the multiple determinants of health.
7. Involves systems development using a cyclical and iterative process
8. Disseminates results to all partners and involves them in the wider dissemination of results
9. Involves a long-term process and commitment to sustainability
10. Openly address issues of race, ethnicity, racism, and other social divides, and embody “cultural humility”16, recognizing that while no one can be truly “competent” in another's culture, we can demonstrate a commitment to self-reflection and critique, working to redress power imbalances and to develop authentic partnerships.
11. Work to assure research rigor and validity but also “broaden the bandwidth and validity”17 to insure that the research question is valid (coming from, or being of importance to the community) and that different “ways of knowing,” including community lay knowledge, are valued alongside more traditional scientific sources of knowledge.17
Following these principles may help strengthen the quality of data and the statistical power of analysis. Yet as Green and Glasgow18 point out (see Chapter 15), although CBPR improves one facet of external validity—its relevance to “end users” of findings in a particular community—CBPR may make it less relevant to other communities. The more we make a study locally relevant, tailoring it to a particular population or community group, the more we make it ungeneralizable beyond that setting and population.18 Such research remains important and relevant to others, however, in that it was made more applicable to typical circumstances, rather than settings that are “artificially constructed and controlled” for academic purposes.18
We turn now to the more specific ways in which CBPR may add value to community-based translational research.
Benefits of CBPR for Improving D&I Research Quality and Relevance
A CBPR orientation to inquiry has the potential to strengthen research quality at each step of the process, many of which have direct relevance for study design and analysis as well as the dissemination and implementation of findings. Although there are fewer examples of CBPR in D&I research specifically, the strengths of CBPR, highlighted below, are increasingly being pointed to as a way to remedy the “lack of “fit” between an intervention/research design on the one hand and, on the other hand, the realities inherent to target and practice settings, and the information needed by policymakers. Such mismatch often leads to “low adoption and implementation.”18
CBPR can help ensure the relevance of the research topic. When “bench-to-bedside” or “bench-to-curbside” translational research is not seen by patients or communities (p.196) as holding relevance for their lives and contexts, even the most elegant of research designs may fail to achieve their intended effects. Although the far end of the CBPR continuum involves communities or other stakeholders in identifying the topic to be studied, engaging such partners early in the process can help to ensure that even investigator-driven research is locally relevant and likely to yield useful results. Stanford University's Chronic Disease Self-Management Program has been tested through numerous peer-reviewed RCTs,19 – 21 yet whether this program would have relevance to Native Americans, who are believed to have the nation's highest rates of diabetes,22 was open to question. Working closely with a Community Advisory Board of Native Americans with diabetes in Santa Clara County, California, Jernigan and her colleagues determined that the program did have relevance but would need to be adapted, for example, in beginning each weekly session with a blessing and smudging ceremony, increasing session length to allow time for storytelling, and incorporating the image of a dream catcher into the program's visual of the symptom cycle.23 With these changes, and led by Native American peer educators, the pilot program had a 100% retention rate, with significant changes seen in a variety of disease symptoms and self-management behaviors. Based on this success, the program was adapted for dissemination over the Internet24— a medium widely used by Native Americans across the United States since a major, cross-tribal newspaper is most easily accessed online.
CBPR can enhance the quality, validity, sensitivity, and practicality of research instruments by involving the local knowledge of community members. Surveys and other research instruments that lack cultural sensitivity or that appear naive or ill suited to the community being studied can reinforce feelings of disconnection and sometimes even be hurtful or insulting. Community insights into how to rephrase questions, or what type of research instrument may be best suited to a given community (e.g., focus groups versus in-depth interviews) can improve recruitment and retention9 – 12 and, as Cargo and Mercer11 point out, help in “reducing measurement error from survey and interview questions that are not culturally aligned.” In the Healthy Homes Project in Seattle–Kings County, Washington, Community Health Workers administering a standard baseline survey to assess exposure to indoor asthma triggers noted that, despite earlier pretesting, questions about whether residents smoked at home were not sensitive enough to pick up whether or not others were smoking inside the house, and survey modifications resulted. As the study's epidemiologists pointed out, “Any loss in the ability to make ‘pure’ baseline and exit comparisons may have been outweighed by the higher quality of the exit data” as a result of community partner input.25 The integration of different types of information and knowledge, through CBPR's inclusion of local knowledge and multiple stakeholder perspectives, is particularly relevant to D&I research, which, as Glasgow and Emmons (2007) and Green (2001) point out, has largely employed “ . . . a limited and researcher-centered perspective as to what constitute ‘evidence.’ ”6 , 26
CBPR can enhance the likelihood of overcoming the distrust of research by communities that traditionally have been the “subjects” of such research by bringing together partners with different skills, knowledge, and expertise to address complex problems. Deloria27 coined the term “helicopter research” in reference to the fact that in “Indian country,” outside researchers often enter to collect their data and (p.197) then leave, offering nothing in return to local community members. Although such an approach has been noted—and criticized—by many communities, it is particularly problematic with respect to marginalized groups and populations for whom decades and sometimes centuries of oppression, including unethical research, at the hands of the dominant population have engendered deep distrust.12 , 28
By increasing the relevance of research interventions, CBPR can increase the likelihood of success. When community input is earnestly sought and valued, interventions may deviate from what the outside researchers originally had in mind.29 Yet such changes may have positive effects for relevance and adaptation, improving external validity in the process.28 Furthermore, participation of community members, including practitioners and policymakers, can enhance the probability of an intervention's adoption, “closing the gap between discovery and delivery.”18 Community Partners in Care, an RCT to improve the relevance of quality improvement approaches to depression care among African Americans, was planned using a community-partnered participatory research approach30 and is now being implemented in Los Angeles, California, following these same participatory research principles and practices.31 The inclusion of multiple stakeholders, among them a wide network of agencies, policymakers, and the arts community (since community partners felt the arts would provide culturally appropriate avenues for opening discussion about the stigma of depression), led to a robust community engagement model whose efficacy compared to more traditional resources for services models and is in the final stages of testing.31
CBPR can improve data analysis and interpretation by enhancing our understanding of the meaning of study findings through the contribution of lay knowledge. Although community members often have neither the time for nor the interest in being engaged in hands-on data analysis,12 , 15 , 32 their help in reviewing and interpreting preliminary findings may add important nuance and deeper understandings of study results.
CBPR can improve the potential for disseminating findings to diverse audiences and translating evidence-based research into sustainable changes in programs, practices, and policies. Publication of CBPR translational research studies in peer-reviewed journals is critical, particularly since to date, the number of such publications that present their methodological approaches in detail and can demonstrate significant health outcomes, remains small.11 , 33 Community and other stakeholder partners, however, can identify additional dissemination channels (e.g., ethnic media and relevant community events) as well as strategies to more effectively reach key community members and decision makers with the findings. A number of CBPR projects also have created workbooks or replication manuals to assist other partnerships interested in adapting and utilizing their approaches, and many of these, along with myriad other resources, are now available online through the Community Tool Box (http://ctb.ku.edu) and Community-Campus Partnerships for Health (www.ccph.info). The combination of community engagement and relevant scientific research further can pack substantial political punch, helping to effect policy and systems-level changes conducive to more health-promoting environments.34 A successful CBPR effort in a low-resource San Francisco neighborhood with little access to fresh fruits and vegetables not only led to the creation of a local “Good Neighbor (p.198) Program” offering incentives to local stores that agreed to stock more fresh fruits and vegetables, but it also played a key role in helping secure the passage of legislation for a statewide demonstration program to help replicate this approach in other communities.34 – 36 Although no funding was allocated after the measure was signed into law, a subsequent amendment for a public-private partnership will enable the program to be taken to scale.
Ethical and Methodological Challenges
Although CBPR can indeed help enhance the quality and relevance of translational research, including making substantial contributions to research design and analysis, this approach also raises difficult ethical and methodological challenges that merit attention.12 , 14 , 29 Community and other stakeholders may make recommendations that would weaken the rigor of study instruments (e.g., altering validated scales) or propose changes in study design that may weaken the science, for example, when community members object to an RCT or staggered design that may be needed to prove whether an intervention had an effect. To avoid such difficulties, many partnerships now begin by developing memorandums of understanding (MOUs) and/or holding colearning workshops where the meaning of terms like validity and research rigor are explored from both academic and community perspectives, and decisions are made ahead of time about how difficult issues will be handled. Partnerships further have benefited by the development of tools, such as Mercer et al.'s37 reliability-tested guidelines for assessing partnership process and progress along multiple dimensions, and tools developed by Israel and her colleagues,38 , 39 which also have been widely adapted and used.
When trained community members are engaged in conducting interviews or administering surveys, perceptions of bias also may be raised. In communities characterized by high levels of distrust borne of years of discriminatory treatment, however, as in some rural parts of the Southern United States, involving outside researchers in face-to-face data gathering may be unrealistic, especially initially. Creative ways of dealing with such problems, such as having residents of neighboring communities accompany and introduce university researchers in data collection activities (e.g., door-to-door surveys) may help in building trust and increasing participation rates.40
In the data interpretation and dissemination phases of translational research, ethical and methodological challenges also may arise if community partners are perceived as having an “axe to grind” that could lead them to present findings selectively to further their community's best interests. Additionally, if data emerge that could cast the community in an unfavorable light,41 thorny questions of community ownership and decision-making processes may ensue. As in other stages of the research process, colearning sessions and trainings on the importance of accurately collecting, analyzing, and reporting findings, as well as frank discussions of community and outside researcher roles and responsibilities in this regard, are critical. At the same time, instruments such as Flicker et al.'s41 recommendations for Institutional Review Boards reviewing participatory research may be useful to CBPR partnerships in asking hard questions up front, including, for example, “What (p.199) will be done if findings emerge that are unflattering to the community?” As noted earlier, process appraisal instruments that CBPR partnerships can use throughout the research process37 – 39 and careful MOUs between partners can aid in preventing or openly confronting such challenges when they arise. Several ethical and methodological challenges and how they were dealt with are highlighted in the case study that follows.
Cbpr Case Example: The San Francisco Chinatown Restaurant Worker Study
The Chinatown Restaurant Worker Study is an ongoing CBPR effort to examine and address the health and working conditions of restaurant workers in San Francisco's Chinatown. Nationwide, workers in the restaurant industry face high rates of injury and other challenges such as low wages and few benefits, limited opportunities for upward mobility or wage increases, and other types of occupational injustice.42 – 44 Studying the relationships between restaurant work and worker health is particularly important for immigrant workers who comprise a large portion of this workforce and who may experience disproportionately greater rates of illness and injury due to immigration concerns, language barriers, and lack of awareness of U.S. workplace regulations.42 Given that restaurants are the largest employer of Chinese immigrants,45 one of the largest and most rapidly growing immigrant populations in the United States research and intervention efforts with Chinese restaurant workers are critical. San Francisco's Chinatown, home to almost 5,000 people and more than 100 restaurants, offered an important setting in which to study and address the working conditions and health of this important community.
Building on strong mutual interest in promoting the health and welfare of Chinatown restaurant workers, a partnership was developed in 2007 comprised of a community-based organization (the Chinese Progressive Association [CPA]), two universities (the University of California, Berkeley School of Public Health, including UC Berkeley's Labor Occupational Health Program [LOHP], and the University of California San Francisco School of Medicine), and a local health department (the San Francisco Department of Public Health's Occupational and Environmental Health Section). Many of the partners had worked previously together on other CBPR projects, and the project coordinator (a university partner) was a founding member of CPA and a former resident of Chinatown. These existing relationships greatly facilitated the establishment of the partnership as well as initial trust between partners.
The community-university-health department collaborative aimed to follow CBPR principles3 , 12 and included an ongoing participatory evaluation of partnership process and effectiveness.46 A 12-member Steering Committee, comprised of representatives from each partner organization, was formed to serve as the project's primary coordinating and decision-making body. A six-member Restaurant Worker Leadership Group (RWLG) was established with the goal of facilitating in-depth participation from restaurant workers throughout all phases of the project.
A CBPR grant from the National Institute for Occupational Safety and Health (NIOSH) supported partnership development, an ecologic study of Chinatown (p.200) restaurant worker health and safety, described in more detail below, a participatory evaluation of the partnership, and some dissemination activities. In keeping with its CBPR orientation, the Chinatown project kept the “final” phases of CBPR—dissemination and translation of findings into action—at the forefront of planning from the study's onset. For example, the team sought additional funding early in project from a large philanthropic organization, The California Endowment, which supports and encourages CBPR and policy-level intervention. This additional funding enabled the partnership to more broadly disseminate study findings and translate research findings into programs and policies to promote worker health. The community partner's long history and success in organizing and advocacy made CPA well suited to lead these expanded dissemination and translational activities, and it served as the lead agency on the California Endowment grant.
Participation in Study Design
Partners’ collaboration in study design began during grant writing and intensified once study funds, the Steering Committee, and the RWLG were all in place. Consistent with partners’ values and CBPR principles and recognizing the multilevel nature of factors influencing restaurant worker health, the Chinatown study employed an ecologic design. The study was comprised of a community-based cross-sectional survey of Chinatown restaurant workers, conducted by trained worker partners. It also included standardized observations of hazards to workers in Chinatown restaurants conducted by health department staff during food safety inspections. All partners, including RWLG members, actively participated in working groups to develop data collection protocols and study instruments. Involving all partners in study development required significant time and commitment from all partners. In addition, considerable resources were needed for critical services such as translation of documents into Chinese and English and simultaneous interpretation into Cantonese or English at project team meetings.
To more effectively involve RWLG members, Chinatown restaurant workers with no prior research experience, an eight-week training was held at the beginning of the study on topics such as workplace health and safety, workers’ rights, research goals, and research-related topics such as confidentiality, informed consent, and survey administration. After the initial training, the RWLG met biweekly to provide feedback on study instruments, develop a recruitment plan for the worker survey, and prepare to pilot-test the survey instrument. Interactive activities such as risk mapping,47 – 49 neighborhood mapping, and mock food inspections in a simulated kitchen were used to enhance workers’ participation.
The instruments developed by the collaborative are described below with some examples of how the involvement of diverse partners improved their quality, and thereby enhanced their application and outcomes.
A standardized questionnaire was developed to measure Chinatown restaurant workers’ health and work experiences. A draft instrument, created with inputs from all partners, was reviewed and revised by the RWLG and then finalized after (p.201) piloting with 15 restaurant workers. The participation of CPA in the development of the questionnaire resulted in many additions to the original draft created by university partners. These included questions designed to learn more about the broader context of workers’ lives (e.g., wages, housing, health and social service utilization, and workers’ civic engagement). The RWLG's participation in survey development resulted in some important wording additions to standardized scales (e.g., explaining such idioms as “butterflies in my stomach”), as well as new questions to document previously ignored hardships in their work environments such as harassment, violence, and wage and tip theft.46
Procedures for survey recruitment and administration similarly were collaboratively developed by all partners. The participation of CPA and the RWLG, in particular, was critical for anticipating possible risks to participants (e.g., worker retaliation) and developing a protocol that assuaged participants’ fears and safeguarded their identities. Members of the RWLG and 17 additional community members who were hired as surveyors went through intensive training in informed consent and study procedures prior to their involvement in recruiting participants and administering the survey.
A 13-item observational survey, the Restaurant Worker Safety Checklist, was developed to collect restaurant-level information about the presence of required labor law postings, occupational hazards, and safety measures, equipment, and behaviors during regularly scheduled food service inspections. The health department partner, SFDPH, took the lead on the development of the checklist, with additional inputs from community and academic partners and health department food inspection staff. RWLG members’ recommendations resulted in important improvements to the final tool. As they pointed out, for example, the checklist should not only assess whether posters detailing Occupational Safety and Health Administration's (OSHA) regulations and San Francisco's wage ordinance were present but also whether Chinese language versions were posted. Further, the checklist should not just assess whether first aid kits were visible but document whether they were fully stocked (82% of restaurants did not have fully stocked kits).46
SFDPH was solely responsible for the administration of the checklist. However, community and university partners shadowed health department staff during checklist piloting. This enhanced partners’ understanding of the types of challenges present in restaurants that affect both data collection and worker health.
Participation in Data Analysis and Interpretation
The collaborative was successful in exceeding research targets. Surveys were completed with 433 restaurant workers, and observational data using the Restaurant Worker Safety Checklist were collected by health department staff in 98% of Chinatown restaurants that were open for business at the time of data collection (n = 106 of 108 possible restaurants). Once data were collected, the partnership followed an agreed-upon process for preparing and analyzing the findings. This process included all partners and was designed to build upon partners’ existing skills and (p.202) expertise (e.g., data analysis and knowledge of local context) as well as codevelop and enhance additional skills necessary for improved interpretation of study results (e.g., “outsiders’ ” understanding of local culture and context and RWLG members’ knowledge of how to read data tables).
University partners took the lead in preparing and analyzing the survey data, and health department partners did so for the observational restaurant-level data. During the analysis period, preliminary results were routinely shared with all partners through e-mail communications and presentations at Steering Committee, RWLG, and other project meetings. A critical component of the analysis phases of the study was the ongoing involvement of restaurant workers in data interpretation. Six monthly data interpretation workshops were held with the RWLG members at CPA's office in Chinatown. These workshops, conducted in Chinese by CPA staff and the project coordinator with additional support from other university and health department partners, employed hands-on learning to teach RWLG members to speak “data language” and to facilitate their participation in the interpretation of checklist and survey findings.
RWLG members provided many insights into the data not originally apparent to other partners. For example, when considering findings indicating that cooks wore long-sleeved shirts or cook jackets in only 10% of restaurant observations conducted,50 RWLG members suggested that, in addition to the high kitchen temperatures, this was likely due to Chinese male cooks’ viewing of burns and cuts as “badges of honor.”46 Similarly, RWLG members helped identify and provide context for some survey findings that they believed to be overreported due to workers’ fears of employer retaliation or misinterpretation. For example, the RWLG doubted that 58% of restaurant worker respondents actually received paid sick days, as reported, explaining, “When people ask for a day off, they work another day later . . . workers often understand this [having to make up a day for taking a sick day] as sick leave.” Similarly, the RWLG explained that many of the statistics related to health, such general health status (SF-36) and work-related injuries, were likely underestimated due to the fact that many workers would only report major problems “like cancer.” One member summed up this phenomenon, saying that Chinese workers think that “unless you're really sick, you're healthy.” RWLG members felt that the same phenomenon resulted in an underreporting of abusive treatment at work such as being yelled at (reported by 42%). RWLG felt that those responding in the affirmative were probably only those for whom the “yelling had made them cry,” explaining that they are constantly being yelled at by their supervisors.
The in-depth participation of all partners in data analysis and interpretation resulted in all partners, including restaurant workers themselves, having a detailed knowledge of study findings. This greatly enhanced equitable and high-level involvement in dissemination of study results as well as the use of data in the creation of translational strategies, discussed below.
Community and stakeholder participation in disseminating and translating results into programs and policies
In contrast to the research phase of the study, in which the researchers, in many ways, took the lead on guiding activities, the community and health department (p.203) partners are leading in the “final phase” of the Chinatown CBPR effort. The partnership's dissemination plan includes sharing study findings through both scientific and lay/ethic media as well as targeted meetings with restaurant workers, restaurant owners, and key policy and decision makers. Recognizing that diverse purposes and audiences would require different types of communications, the partnership created several different reports to share findings: (1) a report of worker survey findings authored by university partners;51 (2) a report of restaurant observation findings written by health department partners;50 and (3) a comprehensive, visually appealing, and action-oriented report authored by CPA52 that draws from the two previous reports to lay out a vision for improving working conditions for a healthy Chinatown. The collaborative's dissemination activities, like other CBPR efforts,53 , 54 are guided by agreements regarding publication and presentation established early in the study. These guidelines emphasize coownership of data and coauthorship. To date, all partners have participated in presenting study findings and have served as coauthors on the peer-reviewed papers published to date.46 , 55
Multiple efforts are currently underway to translate study finding into sustainable improvements for Chinatown restaurant workers. As a result of the significant lack of labor law posting documented in this study (e.g., only 15% of observed restaurants had posted workers’ compensation in English and 8% in Chinese; just 24% had the city's sick leave regulation posted in English and 23% in Chinese), and the results of a subsequent study to examine compliance with labor law posting in restaurants within and beyond Chinatown, SFDPH began requiring proof of workers’ compensation insurance coverage for all new and change-of-business health permits. The health department is also taking additional steps to assess citywide compliance with these policies. For example, as a result of study findings that 50% of Chinatown restaurant workers did not receive minimum wage, 52% reported not receiving paid sick leave, 40% didn't received mandated work breaks, and 97% did not receive the city's required health care coverage, SFDPH sent formal letters to regulatory bodies such as the local Office of Labor Standards Enforcement (OSLE) to share study findings and request collaboration in improving enforcement of these laws. As a result, OLSE and SFDPH are exploring mechanisms to improve violator identification and enforcement.55 Additionally, the health department has obtained new funding with study partner LOHP to explore feasible ways to involve food safety inspectors (who have legal access to restaurant environments) in the promotion of workers’ health.
CPA is leading a number of efforts to implement or support action interventions based on study findings. Educational activities with restaurant workers, which began early in the project with the RWLG, have been scaled up in number and scope, to reflect study findings. Efforts to educate workers are a result of our finding that 64% of workers surveyed did not receive any mandated health and safety training at work, and documentation, through the study, of wage theft and many other regulatory violations. Ongoing educational efforts include Worker Teas held monthly at CPA, educational exchanges with other workers and worker rights groups around San Francisco and nationally (e.g., at a large workers’ Social Forum), and others. Since no study of this magnitude had previously been conducted with Chinese restaurant workers in the United States, and since restaurants remain the largest employer of immigrant (p.204) Chinese workers throughout the nation study findings have been cited by worker centers and other organizations in New York, Los Angeles, and other major cities.
Additionally, in consultation with project partners and other stakeholders, and based in part on study findings, CPA has developed a key set of recommend actions to improve the health and well-being of Chinatown restaurant workers. These include: (1) convening community stakeholder roundtables to develop solutions for creating healthy jobs and a healthier community; (2) strengthening government enforcement of labor, health, and safety laws; and (3) significantly increasing investments in healthy economic development and responsible employment practices in Chinatown. Along with these general recommendations, CPA has drawn up a number of municipal policy actions. The above-mentioned report authored by CPA details these recommendations and the associated policy agenda.52 The report was launched in September 2010, with many project partners and close to 170 community members, neighborhood organization representatives, media personnel and policymakers in attendance. With the launch, which received widespread media coverage in local newspapers, television and radio programs, as well as on line, CPA initiated a Low Wage Worker Bill of Rights organizing campaign to create support for policy change. At a kickoff event on the steps of the City Hall in Spring 2011, CPA and its partner organization, the Progressive Workers Alliance, introduced a multipronged policy approach to preventing or redressing wage theft and related violations, as well as improving worker education and protection for employer retaliation. When speaking on the bill he cosponsored, a prominent local Supervisor remarked, “I am proud to be introducing local legislation that is drawn from action-based research and bottom-up grassroots organizing that will help strengthen labor law enforcement in San Francisco and give workers a meaningful voice in stopping wage theft in our city.” The event, attended by many community, academic, and health department partners from the study, as well as policymakers and others, was captured on several nightly news programs, and in a prominent story in the leading local newspaper the next day. It also was widely covered in the ethnic media locally and beyond.
Parallel dissemination activities underway by members of the partnership include efforts by the Labor Occupational Health Program and the San Francisco Department of Health to investigate the potential for disseminating the Restaurant Worker Checklist to other health departments within California and nationally. The sharing of the final report of restaurant observation findings and the project's observational checklist tool through the SFDPH's website (http://www.sfphes.org/publications/Restaurant_Health_Safety_Checklist.pdf), as well as an article about the check list in Public Health Reports,55 a journal that is received by many health departments and public health practitioners around the nation, also are expected to help facilitate the dissemination process.
Lessons Learned And Implications For Dissemination And Implementation Research
The Chinatown Restaurant Worker Project described above in many ways exemplifies the potential value of CBPR for improving the relevance of research and (p.205) enhancing the dissemination and implementation of findings through collaborative design, analysis, dissemination, and research translation. Drawing on this and other participatory research case studies, as well as the now substantial body of literature in the field of CBPR, we present key lessons learned and their implications for research dissemination and implementation.
Through the codevelopment of research priorities and design with community and other stakeholders, CBPR can help ensure that the research is relevant to the community, potentially helping improve its external validity.56 Further, such cogeneration of research topic and study design can enhance buy-in and shared commitment on the part of all partners in moving from research to action. In the Chinatown CBPR study and numerous others,11 , 12 , 15 , 25 , 57 the participation of community partners in the design of research instruments resulted in the generation of new survey items and improved data-gathering tools that in turn proved critical to a fuller understanding of the topic under investigation—and the design of subsequent research-based action.
Inclusion of multiple stakeholders, including but not limited to a strong community partner, also is advised. As illustrated in the Chinatown case study, having as a partner a local health department can be important for gaining entrée into the community and to environments (i.e., worksites) that otherwise would likely be “off limits.” Further, including as partners one or more policymakers, and/or having a strong policy mentor, may be critical for those partnerships wishing to help affect broader, systems-level change. In West Oakland, California, a CBPR partnership to study and address the large number of diesel trucks driving through and idling in a low-income portside community, profited early on by including a local city council member as a policy mentor and active partner. By holding monthly meetings in the councilor's office, where study findings were discussed and a proposed new truck route ordinance designed, the partnership was able to get many disparate stakeholders to the table, and several subsequent policy wins followed in part as a result.58
The special benefits of a participatory approach for dissemination of study findings through diverse channels and to multiple audiences also should be underscored. Publication of study methods and findings in respected peer-reviewed journals is, of course, critical, both for extending the study's reach and underscoring its scientific merit. Yet as Canadian scholar Dennis Raphel is fond of asking, “If an article is published in Social Science and Medicine but nobody reads it, does it exist?” A prolific scholar himself, Raphel's message is not to avoid publishing, but rather to insure that we do not stop there. Many CBPR partnerships, including the Chinatown study, have a special subcommittee that helps to ensure the wide dissemination of findings through a diversified strategy. Proposed journal articles and abstracts for presentations at professional meetings, typically with community partner coauthors, are reviewed by these committees. But attention also is focused on effective use of the mass media (including local language newspapers), presentations at community forums, development of policy briefs based on study findings, and other nonacademic means of disseminating findings to promote education and action. The involvement of multiple stakeholders, especially those “in” the community, increases the capacity to disseminate findings in a meaningful and culturally appropriate way.
(p.206) Furthermore, as in the Chinatown project, the inclusion of a health department may enhance the partnerships’ ability to speak the “language” of, and have greater success in engaging, regulators and policymakers in organizational- and policy-level interventions to address research findings. As an inherently collaborative and colearning process, CBPR enhances the expertise and capacity of all partners for culturally appropriate research and prevention that addresses contextual factors. While university partners bring research expertise to the table, the inclusion of community and health department partners, who often are more skilled in organizing and policy advocacy than their academic counterparts, can be critical for ensuring that findings are translated more broadly into innovative actions to address social, economic, and political determinants of health. Such actions may be more likely to result in sustainable and long-term impacts than more traditional health and public health interventions.34 , 59
Ethical and methodological challenges will almost invariably emerge in the course of a CBPR effort—a fact that underscores the importance of preparing, in advance via MOUs or less formal mechanisms, as well as encouraging ongoing dialogue around tough issues such as funding and workload equity. Although as noted above, useful instruments have been developed to help guide such discussions,37 – 39 , 41 the unique composition and needs of each partnership suggest the utility of tailoring such tools, or devising new ones, to best serve a particular partnership.
Among the most challenging aspects of CBPR, with special relevance to D&I research, is that this orientation to research requires a long-term commitment from all partners. In addition to the added time involved in front-end partnership building and maintenance (e.g., involving numerous steering committee and advisory board meetings, community meetings, retreats, etc.), the translation and action phases of such work may often not take place until after the funded research period. In a Harlem, New York–based CBPR effort to promote the successful community reintegration of former substance-abusing inmates, several key policy victories were achieved only well after the project's federal support had ended. Had the Harlem Community and Academic Partnership members not continued to work together, the impressive policy changes to which they contributed, including legislation mandating discharge planning services, help finding housing and drug treatment services, and reinstatement of Medicaid coverage immediately upon release from prison or jail, might never have been achieved.57 , 60 Complicated issues take time to understand and address, and rarely align neatly with academic or community partner timetables.
Successful translation and use of CBPR findings also may require obtaining additional funding, including some from nongovernmental and other sources that can constrict or preclude advocacy on behalf of relevant legislation. In the Chinatown case study, substantial supplemental funding from a large and progressive foundation committed to policy-level change, with the community partner as the lead agency, proved particularly suited for translating study finding into policy-level interventions. Yet philanthropic organizations differ in their support for policy-related activities, and CBPR partnerships whose goals include action on the legislative level should determine in advance whether and to what extent their funding source(s) will support and/or even allow such activity.
An inherently action-focused research orientation, CBPR is particularly well suited to D&I research. As suggested in this chapter, although CBPR is not relevant in all or even most research contexts, when there is goodness-of-fit between this orientation to research and a proposed study, significant value can be added to both the processes and outcomes of the research. Prominent among the latter are enhanced dissemination and implementation of findings through the authentic engagement of community partners and other stakeholders throughout. In our discussion of CBPR, and our use of the Chinatown Restaurant Worker Study to illustrate its principles in action, we have highlighted many of the benefits CBPR can offer to research and its dissemination and implementation. Drawing on this and other CBPR case studies and literature, we also have suggested a number of implications that CBPR holds for dissemination and implementation research and bridging the gap between research and practice.
In concluding, we would like to emphasize an additional and important strength that CBPR offers for reducing health disparities (as discussed in Chapter 22). When conducted in accordance with its key tenets and principles, CBPR can be an important paradigm for promoting not only health equity in the sense of distributive justice, but also the “procedural justice”61 necessary for real change to take place, and be sustained over time. Procedural justice has been described as involving “equitable processes through which low-income communities of color, rural residents, and other marginalized groups can have a seat at the table—and stay at the table—having a real voice in decision making affecting their lives.”34 In the words of one RWLG member from the Chinatown study, involvement in a CBPR study and its subsequent translational efforts can yield “courage to confront problems in [our] community.” Through reciprocal capacity building of community, university, and other partners; its establishment of “structures for participation”; and its provision, especially for underrepresented communities, of a “place” for their voices to be heard and a way to make change,33 CBPR can be a potent mechanism for addressing some of the social and other inequalities at the heart of many health disparities.
Cargo M, Mercer SL. The value and challenges of participatory research: strengthening its practice. Annu Rev Public Health. 2008;29:325–350.
This thorough and sophisticated article on CBPR in the health field provides a critical review of the literature, followed by an “integrative practice framework” highlighting key domains including values and drivers (such as knowledge transfer and self- determination), partnership processes, and the interpretation and application of research outcomes.
Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health. 1998;19:173–202.
The single most cited paper in CBPR in the health field. This paper introduces CBPR, its core principles, as well as some of the challenges entailed in their implementation. In addition to providing a review of the literature, this work uses early lessons of the Detroit Community-Academic Research Center to explicate CBPR principles and their implementation.
Edited by four leaders in the field, this book provides an introduction to a wide range of methods in CBPR, including topics such as partnership development and maintenance; community assessment and diagnosis; definition of issues, documentation of partnership processes, and the interpretation, dissemination, and application of research findings. Its 17 chapters and 16 appendixes include many tools and examples from the Detroit Community-Academic Research Center and elsewhere to illustrate the methodological approaches and other topics under consideration.
Minkler M, Wallerstein N. Community-Based Participatory Research for Health: From Process to Outcomes. 2nd ed. San Francisco, CA: Jossey-Bass;2008.
The first major volume on CBPR in the health field in the United States, this coedited text, now in its second edition, includes 21 chapters and 12 appendixes covering a wide range of theoretical, methodological, ethical, and practical issues in CBPR, with a special emphasis on policy and other health-related outcomes. Key topics include the theoretical and practice roots of CBPR, issues of power and trust in working cross-culturally, ethical and methodological challenges, participatory evaluation, and CBPR and policy. Numerous case studies and practical tools are included, for example, reliability-tested guidelines for appraising participatory research and guidelines for institutional review boards (IRBs).
O’Fallon L, Dearry A. Community-based participatory research as a tool to advance environmental health sciences. Environ Health Perspect. 2002;110(Suppl 2): S155–159.
Although specifically focused on CBPR as it relates to environmental health and environmental justice, this seminal article provides a thoughtful laying out of the ways in which community engagement strengthens research processes and outcomes, as well as the challenges inherent in this approach. Several case studies from environmental health are used to illustrate the issues raised, and the article remains widely cited in public health and other disciplines.
Selected Websites and Tools
The Community Tool Box (http://ctb.ku.edu). Created by the Work Group for Community Health and Development at the University of Kansas, and over 6,000 pages in length, this well-organized website offers numerous tools for participatory community assessment and evaluation, as well as other aspects of CBPR and related approaches.
Community-Campus Partnerships for Health (www.ccph.info). This site is a portal to a wide array of resources for partnerships undertaking CBPR, including sample memoranda of understanding, tools for collaborative asset and risk mapping, research dissemination, and articles and workbooks on the translation of findings into policy and practice change.
1. Green L, George M, Daniel M, et al. Study of participatory research in health promotion. Ottawa: The Royal Society of Canada;1994.
2. Hall B. From margins to center: The development and purpose of participatory action research. Am Sociol. 1992;23(4):15–28.
3. Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health. 1998;19:173–202.
5. Am. IMCQHC. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: Natl. Acad. Press;2001.
5. Glasgow RE, Emmons KM. How can we increase translation of research into practice? Types of evidence needed. Annu Rev Public Health. 2007;28:413–433.
7. Horowitz CR, Robinson M, Seifer S. Community-based participatory research from the margin to the mainstream: are researchers prepared? Circulation. 2009;119: 2633–2642.
8. Biggs SD. Resource-Poor Farmer Participation in Research: A Synthesis of Experiences from Nine National Agricultural Research Systems. The Hague, Netherlands: ISNAR;1989.
9. Kaplan SA, Dillman KN, Calman N, Billings J. Opening doors and building capacity: employing a community-based approach to surveying. J Urban Heatlh. 2004;81: 291–300.
10. O’Fallon L, Dearry A. Community-based participatory research as a tool to advance environmental health sciences. Environ Health Perspect. 2002;110(Suppl 2): S155–159.
11. Cargo M, Mercer SL. The value and challenges of participatory research: strengthening its practice. Annu Rev Public Health. 2008;29:325–350.
12. Minkler M, Wallerstein N. Community-Based Participatory Research for Health: From Process to Outcomes. 2nd ed. San Francisco, CA: Jossey-Bass; 2008.
13. W.K. Kellogg Foundation Community Health Scholars Program. Stories of Impact [brochure]. Ann Arbor, MI: University of Michigan, School of Public Health, Community Health Scholars Program, National Program Office; 2001.
14. Minkler M. Community based research partnerships: Challenges and Opportunities. J Urban Health. 2005;82(2):3–12.
15. Israel BA, Eng E, Schultz A, Parker E. Methods in Community-Based Participatory Research for Health. San Francisco, CA: Jossey-Bass; 2005.
16. Tervalon M, Murray-Garcia J. Cultural humility versus cultural competence: a critical distinction in defining physician training outcomes in multicultural education. J Health Care Poor Underserved. May 1998;9(2): 117–125.
17. Reason P, Bradbury H. Handbook of Action research: Participative Inquiry and Practice (Concise ed.). Thousand Oaks, CA: Sage; 2008.
18. Schillinger D. An introduction to effectiveness, dissemination and implementation research. In: P. Fleisher and E. Goldstein, eds. UCSF Clinical and Translational Science Institute (CTSIResource Manuals and Guides to Community-Engaged Research, P. Fleisher, ed.): Published by Clinical Translational Science Institute Community Engagement Program, University of California San Francisco; 2010. Accessed December 19, 2010. http://accelerate.ucsf.edu/files/CE/edi_introguide.pdf
19. Lorig KR, Ritter P, Stewart AL, et al. Chronic disease self-management program: 2-year health status and health care utilization outcomes. Med Care. Nov 2001;39(11): 1217–1223.
20. Lorig KR, Sobel DS, Ritter PL, Laurent D, Hobbs M. Effect of a self-management program on patients with chronic disease. Eff Clin Pract. Nov–Dec 2001; 4(6): 256–262.
21. Lorig KR, Sobel DS, Stewart AL, et al. Evidence suggesting that a chronic disease self-management program can improve health status while reducing hospitalization: a randomized trial. Med Care. Jan 1999;37(1): 5–14.
23. Jernigan VB. Community-based participatory research with Native American communities: the Chronic Disease Self-Management Program. Health Promot Pract. 2010;11(6):888–899.
24. Jernigan VB, Lorig K. The Internet Diabetes Self-Management Workshop for American Indians and Alaska Natives. Health Promot Pract. 2011;12(2):261–70.
25. Kreiger J, Allen C, Roberts J, Ross L, Takaro T. What's with the wheezing? Methods used by the Seattle-King County Healthy Homes Project to assess exposure to indoor asthma triggers. In: Israel BA, Eng E, Schultz A, Parker E, eds. Methods in Community-Based Participatory Research for Health. Vol. 230–250. San Francisco, CA: Jossey-Bass; 2005.
26. Green L. From research to “best practices” in other settings and populations. Am. J. Health Behav. 2001;25:165–178.
27. Deloria V, Jr. Commentary: research, redskins, and reality. Am Indian Q. 1991;15(4): 457–468.
28. Green LW, Mercer SL. Can public health researchers and agencies reconcile the push from funding bodies and the pull from communities? Am J Public Health. 2001;9(12):1926–1929.
29. Buchanan D, Miller F, Wallerstein N. Ethical issues in community-based participatory research: balancing rigorous research with community participation in community intervention studies. Progr Community Health Partnersh. 2007;1.2:153–160.
30. Jones D, Franklin C, Butler BT, Williams P, Wells KB, Rodriguez MA. The Building Wellness project: a case history of partnership, power sharing, and compromise. Ethn Dis. Winter 2006;16(1 Suppl 1):S54–66.
31. Bowen C, Jones L, Dixon EL, Miranda J, Wells K. Using a community partnered participatory research approach to implement a randomized controlled trial: planning community partners in care J Health Care Poor Underserved. 2009;21(780–795).
32. Cashman SB, Adeky S, Allen A, et al. Analyzing and interpreting data with communities. In: Minkler M, Wallerstein N, eds. Community-Based Participatory Research for Health: From Process to Outcome. Second ed. San Francisco, CA: Jossey-Bass; 2008:285–301.
33. Viswanathan M, Ammerman A, Eng E, et al. Community-Based Participatory Research: Assessing the Evidence. Summary, Evidence Report/Technology Assessment No. 99. Rockville, MD: Agency for Healthcare Research and Quality;2004.
34. Minkler M. Linking science and policy through community-based participatory research to eliminate health disparities. Am J Public Health. 2010;100(Suppl 1): S81–87.
35. Hennessey Lavery S, Smith ML, Esparza AA, Hrushow A, Moore M, Reed DF. The community action model: a community-driven model designed to address disparities in health. Am J Public Health. Apr 2005;95(4): 611–616.
36. Vasquez VB, Lanza D, Hennessey-Lavery S, Facente S, Halpin HA, Minkler M. Addressing food security through public policy action in a community-based participatory research partnership. Health Promot Pract. Oct 2007;8(4): 342–349.
37. Mercer SL, Green LW, Cargo M, et al. Reliability-tested guidelines for assessing participatory research projects. In: Minkler M, Wallerstein N, eds. Community-Based Participatory Research for Health from Process to Outcome. San Franciso, CA: Jossey-Bass; 2008:408–418.
(p.211) 38. Israel BA, Lantz PM, McGranaghan RJ, Kerr D, Guzman JR. Detroit Community-Academic Urban Research Center In-Depth, Semistructured Interview Protocol for Board Evaluation, 1996–2002. In: Israel BA, Eng E, Schultz A, Parker E, eds. Methods in Community-Based Participatory Research for Health. San Francisco, CA: Jossey-Bass; 2005:425–429.
39. Israel BA, Lantz PM, McGranaghan RJ, Kerr D, Guzman JR. Detroit Community-Academic Urban Research Center Closed-Ended Survey Questionnaire for Board Evaluation, 1997–2002. In: Israel BA, Eng E, Schultz A, Parker E, eds. Methods in Community-Based Participatory Research for Health. San Francisco, CA: Jossey-Bass; 2005:430–433.
40. Farquhar S, Wing S. Methodological and ethical considerations in community-driven environmental justice research: two case studies from rural North Carolina. In: Minkler M, Wallerstein N, eds. Community-Based Participatory Reserach: From Process to Outcome. Second ed. San Francisco, CA: Jossey-Bass; 2008: 263–280.
41. Flicker S, Travers R, Guta A, McDonald S, Meagher A. Ethical dilemmas in community-based participatory research: recommendations for institutional review boards. J Urban Health. Jul 2007;84(4): 478–493.
42. Restaurant Opportunities Center of New York (ROC-NY). Behind the Kitchen Door: Pervasive Inequality in New York's Thriving Restaurant Industry. New York: Restaurant Opportunities Center of New York and the New York City Restaurant Industry Coalition; 2005.
43. United ROC. A Summary of Restaurant Industry Summaries in New York, Chicago, Metro Detroit, New Orleans, and Maine: National Executive Summary. New York, Chicago, Detroit, New Orleans, Portland: Restaurant Opportunities Center United; 2010.
44. Webster T. Occupational hazards in eating and drinking places. Compensation and Working Conditions. 2001;2001(Summer):27–33.
45. U.S. Census Bureau (2000). Summary File 3: P49; Sex by Industry for the Employed Civilian Population over 16+ Years. http://factfinder.census.gov/.
46. Minkler M, Tau Lee P, Tom A, et al. Using community-based participatory research to design and initiate a study on immigrant worker health and safety in San Francisco's Chinatown restaurants. American J Ind Med. 2010;53(4):361–371.
47. Mujica J. Coloring the hazards: risk maps research and education to fight health hazards. Am J Ind Med. 1992;22(5):767–770.
48. Brown MP. Worker risk mapping: an education-for-action approach. New Solut. 1995;5:22–30.
49. Brown MP. Risk mapping as a tool for community-based participatory research and organizing. In: Minkler M, Wallerstein N, eds. Community-Based Participatory Research for Health. San Francisco, CA: Jossey-Bass; 2008.
50. San Francisco Department of Public Health. Health and Safety in San Francisco's Chinatown Restaurants: Findings from an Observational Survey. San Francisco, CA: San Francisco Department of Public Health; 2009.
51. Salvatore AL, Krause N. Health and Working Conditions of Restaurant Workers in San Francisco's Chinatown: Report of Survey Findings. UC Berkeley and UC San Francisco; 2010.
52. Chinese Progressive Association. Check Please!: Health and Working Conditions in San Francisco Chinatown Restaurants. San Francisco, CA; 2010.
(p.212) 53. Wing S, Horton RA, Muhammad N, Grant GR, Mansoureh T, Thu K. Integrating epidemiology, education, and organizing for environmental justice: community health effects of industrial hog operations. Am J Public Health. 2008;98(8):1390–1397.
54. Parker E, Israel B, Robins T, et al. Evaluation of community action against asthma: a community health worker intervention to improve children's asthma-related health by reducing household environmental triggers for asthma. Health Education and Behavior. 2008;35(3):376–395.
55. Gaydos M, Bhatia R, Morales A, et al. Promoting health equity and safety in San Francisco's Chinatown restaurants: findings and lessons learned from a pilot observational survey. Public Health Rep. 2011;126(suppl 3):62–69.
56. Green LW, Glasgow RE. Evaluating the relevance, generalization, and applicability of research: issues in external validation and translation methodology. Eval Health Prof. Mar 2006;29(1): 126–153.
57. Minkler M, Breckwich Vásquez V, et al. Promoting Healthy Public Policy Through Community-Based Participatory Research: Ten Case Studies. Oakland, CA: PolicyLink; 2008.
58. Gonzalez P, Minkler M, Gordon M, et al. Community-Based - Participatory Research and Policy Advocacy to Reduce Diesel Exposure in West Oakland, California. Am J Public Health. May 9, 2011. [Epub ahead of print.]
59. Morello-Frosch R, Pastor MJ, Sadd J, Porras C, Prichard M. Citizens, science and data judo: leveraging secondary data analysis to build a community-academic collaborative for environmental justice in Southern California. In: Israel BA, Eng E, Schultz A, Parker E, eds. Methods in Community-Based Participatory Research for Health. San Francisco, CA: Jossey-Bass; 2005:371–393.
60. Minkler M, Freudenberg N. From community-based participatory research to policy change. In: Fitzgerald H, Burack C, Seifer S, eds. Handbook for Engaged Scholarship: Contemporary Landscapes, Future Directions Vol. 1: Institutional Change. East Lansing, MI. Michigan State University; 2010:275–294.
61. Kuehn R. A taxonomy of environmental justice. Environmental Law Reporter. 2000;30(10681–10703).