Primary Care Practice Redesign: Challenges in Improving Behavioral Health Care for a Vulnerable Patient Population

Abstract

This study examined a program focused on integrating mental health in a family medicine practice in an economically challenged urban setting. The program included using a behavioral health technology platform, a behavioral health collaborative composed of community mental health agencies, and a community health worker (CHW). Of the 202 patients screened, 196 were used for analysis; 56% were positive for anxiety, 38% had scores consistent with moderate to severe depression, and 34% were positive for post-traumatic stress disorder. There was a statistically significant difference in the diagnosis of depression when comparing the screened group to a control group. Only 27% of patients followed through with behavioral health referrals despite navigational assistance provided by a CHW and assured access to care through a community agency engaged with the Behavioral Health Alliance. Further qualitative analysis revealed that there were complex patient factors that affected patient decision making regarding follow-up with behavioral health care.

Keywords

primary care behavioral health integrated care underserved populations

Mental health disorders contribute substantively to the disease burden in the United States, with an estimated 20% to 25% of adults experiencing a diagnosable mental illness within a 12-month period.¹ Many mental health problems are associated with chronic illnesses, poor health outcomes, and increased cost of care.^2,3 Minority groups have a disproportionately high burden of disability resulting from mental health disorders.⁴ This disparity can be attributed in part to racial and ethnic disparities, with a lower proportion of these individuals receiving mental health care when compared to the non-Hispanic white population.⁵ Multiple factors hinder access to mental health care, including a lack of awareness about mental health disorders, stigma and cultural influences, and lack of understanding about how to access care.^4,6 Narrowing the gap in mental health care for minority groups is an imperative to advancing health equity and improving population health outcomes. This is true in primary care practices where mental health disorders are most often diagnosed and treated.

The US Preventive Services Task Force (USPSTF) recommends routine screening for depression in the general adult population.⁷ The Task Force noted that screening should be in the context of appropriate systems to ensure accurate diagnosis and coordination of treatment and follow-up. Some primary care settings have integrated behavioral and physical models of care, and they are already engaged in evidence-based practices that address the Task Force recommendations. Yet, two thirds of primary care physicians nationally reported the inability to obtain outpatient mental health services for their patients, thus interfering with their ability to create coordinated or integrated mental health care models for their patients.⁸

In a position statement, the American Academy of Family Physicians recommends several strategies to incorporate mental health services into primary care: utilizing technology, empowering staff, using the well visit as an opportunity to screen patients, and using care management and behavioral health consultation when integration is not feasible.⁹ Considering these recommendations, the research team implemented a quality improvement program focused on integrating mental health in a family medicine practice that serves a racially and ethnically diverse population in an economically challenged urban setting: 67% of the residents met the definition of poverty, 20% were unemployed, and 23% had less than a high school education.¹⁰ The program included a 3-pronged approach, using a comprehensive behavioral health technology platform with a survey, a behavioral health collaborative composed of community mental health agencies, and a community health worker (CHW) who assisted with problems associated with low health literacy, social determinants of health, and patient navigation in a culturally sensitive manner. The purpose of this study was to evaluate the effectiveness of the program on the following end points: (1) the screening process, (2) improvements in rates of the diagnoses of depression, (3) patient follow-up with behavioral health resources, and (4) staff perception of the program.

Methods

A mixed-methods study design was utilized with quantitative and qualitative methods from March 2017 through October 2017. The 2 methods were designed as separate inquiries with the results mixed together during data interpretation to enhance the quantitative findings. The proposal was submitted to the Temple University Institutional Review Board and designated as a quality improvement study.

Setting and Sample

The setting was an academic family medicine faculty practice of the Lewis Katz School of Medicine at Temple University, located in North Philadelphia, Pennsylvania. The practice was selected to test the implementation of a validated mental health screening survey integrated with a routine primary care visit. Two of the 5 physicians in the practice were engaged in the project. The patients were adults with scheduled appointments in the clinic.

Procedures

The quality improvement design centered on a survey completed on an iPad by the patient; an interaction with the physician based on the results during the visit; and, if indicated, a referral to a behavioral health resource. A project team was developed including the primary care physicians, practice administrator, CHWs, and representatives from the Temple Center for Population Health. The project team prepared, developed, and tested visit workflows (Figure 1). The team met weekly to evaluate workflows, quality metrics, and staff feedback, and made course corrections in real time.

Figure 1.

Behavioral health workflow.

Software Platform

An information technology (IT) behavioral health platform, BH-Works (Medical Decision Logic, Inc., Baltimore, Maryland), was used to support the program. This web-based tool contained a comprehensive patient self-reported behavioral health screening survey that has been field tested and found to be feasible and psychometrically strong.¹¹ The tool screened for: depression, generalized anxiety disorder, meirishaslego stress disorder (PTSD), suicidality, substance use disorders, and risk behaviors. The skip logic within BH-Works balanced response burden; patients were only asked additional questions if they had a potential problem in any given area. The screening items were written at a sixth-grade reading level, available in English and Spanish, and automatically scored with a summary immediately available for the provider to use during the visit.

A referral resource list was entered into the BH-Works platform and was available electronically to facilitate provision of outpatient mental health services. The geographic locations of community agency counseling sites were identified through mapping home addresses of the family physicians’ patient panels, selecting counseling sites near areas of concentration. These sites were contacted and invited to join a “Behavioral Health Alliance.” Data on the agencies that agreed to participate in this quality improvement effort were uploaded into the BH-Works platform to facilitate tracking of referrals and follow-up.

Behavioral Health Alliance

The Behavioral Health Alliance, composed of 5 community agencies in 11 locations within a 5-mile radius of the practice, was developed to engage the practice with referral sources in the community. Community agency leaders were invited to meet with project team members and physicians to discuss the project, assess needs, and strengthen working relationships. Referral processes and access to care were reviewed, and potential barriers were discussed at the outset of the project and again at 3 months. This represented a new level of engagement between the health system and community mental health agencies focused on a specific mental health improvement project.

Screening and Referral

The survey was initially offered to every third patient by the registration clerk using a standardized script. For those who agreed, screening was completed using an iPad in the waiting room. A CHW and a private room were available if the patient needed help reading or using the tablet device. Though it was desirable to screen all patients, it was evident that the time required to complete the survey slowed patient flow through the clinic. As a result, the screening cadence was changed to every fifth patient presenting at the registration desk. As each patient enrolled in the study completed the survey on the iPad, the results were electronically collated into a report and uploaded into the electronic medical record (EMR).

The results of the survey were reviewed and assessed by the physician and discussed with the patient during the visit. Referrals to behavioral health services were provided at the discretion of the physician. The referral to a counseling location was generated within BH-Works, and the patient was provided with this information in a printed format. Patients who were referred to community behavioral health resources also were referred to the CHW for assistance with logistics, including transportation and appointment scheduling. However, midway through the project it became clear that many patients were not following through with referrals and had not been seen by behavioral health services. To address this deficit, the CHW workflow was changed to include meeting the patient in the exam room after the physician visit was complete, to create a “warm handoff” to the CHW. Additionally, follow-up phone calls were made by the CHW 2 days after the primary care appointment to ascertain if the patient needed help with making a behavioral health appointment. The CHW reached out again 1 month after the referral was made to explore if the patient had followed up with the referral, and to provide continued support if needed.

Patients who were not surveyed received the usual standard of care, completing a Patient Health Questionnaire-9 at the discretion of the physician during the visit.

Measures and Data Collection

Data collected on the screening process included the following: (1) number of patients screened and time to complete the screen, both available through the BH Works system; (2) number of patients who declined screening as recorded by front desk staff; and (3) number of and reasons for patients who needed assistance to complete the tool as recorded by the CHW. Demographic data including race, ethnicity, age, and marital status were collected from the BH-Works platform.

Screening results within the tool, including responses to each question, were captured in the BH-Works software platform provider summary reports. In order to have a comparative group, the participating physicians served as their own control, comparing Depression Diagnoses International Classification of Diseases, 10th Revision codes that were abstracted from the EMR for all patients in the physicians’ panels over the study period.

To understand the factors affecting successful implementation of the program and to provide contextual information for the quantitative results, a model for the spread and sustainability of innovation in service delivery organizations was used as the theoretical framework for the qualitative inquiry.¹² Throughout the implementation process the lead evaluator attended the weekly quality improvement meetings; made practice observations; and collected field notes. At the completion of the program semistructured interviews were conducted with all of the performance improvement team members. The field notes and interview guide were structured on the key elements of the conceptual model, including system antecedents and readiness, characteristics of the adopters and innovation, communication, and implementation processes.

The qualitative inquiry to understand the low number of patients who were following up with their behavioral health referrals used sensemaking theory.^13,14 A structured question was asked of the patient during the routine follow-up call to understand the influences affecting patients’ decision-making processes. The CHW was trained how to ask and document patient responses, which were collected in a secure work plan document. The documentation was reviewed by the evaluator, who also interviewed the CHWs to gain a better understanding of patient responses.

Data Analyses

Quantitative

The primary clinical outcome measure was the rate of patients diagnosed with depression, comparing patients who were surveyed utilizing the BH-Works tool and those who were not surveyed. Depression was selected because of the USPSTF recommendations for screening for this disorder, and because there are well-defined national and state prevalence rates. To determine the effect of the BH-Works tool on rates of the diagnosis of depression, the research team utilized EMR data to calculate the 2 cohorts’ rates of depression using the Fisher exact test (P < .05) and an odds ratio (OR) with a 95% confidence interval (CI). The consistency or agreement between a positive screen for depression for patients using the BH-Works tool, defined as scoring moderate to severe on the assessment tool, and a diagnosis of depression in the EMR was examined using the McNemar test. The remaining 5 domains of behavioral health disorders were analyzed using descriptive analyses.

Qualitative

In the first qualitative strand of the study exploring the facilitators and barriers to implementing the BH-Works system in a primary care setting, the template style was used to organize the data.¹⁵ The coding scheme was developed a priori using the elements defined in the theoretical framework. The segments were then sorted using NVivo version 9 (QSR International Pty Ltd, Doncaster, Victoria, Australia). Data were displayed to simultaneously analyze the segments from field notes and interview transcripts for connections. This step was followed by immersion/crystallization to identify themes.¹⁶

In the second qualitative strand of the study, data were extracted from both the CHWs’ notes and transcripts regarding patients’ reports on how and why they made decisions related to behavioral health referrals. Data from the CHW interviews and chart reviews were analyzed using the immersion/crystallization technique to identify themes. In both qualitative strands, the results were shared with the project team for member checking. The group critically analyzed the findings, and minor revisions were made based on feedback.

Results

A total of 202 patients completed the screening tool, with 196 patients used for inferential analyses. The number screened represents 19.8% of patients seen during the study period. For comparison, 816 non–BH-Works patients from the same practice and seen by the same physicians from the same time period were included in the analysis. Detailed demographic information for all patients is in Table 1.

Table 1.

Characteristics of Patients Evaluated With and Without Formal Behavioral Health Screening Tool.

	No Screening Tool (%)	Screening Tool (%)
Race
African American/Black	73	75
Asian	2	1.5
White	11	8
Hispanic/Black	5	6
Hispanic/other	5	5
Other	4	4.5
Sex
Female	59	59
Male	41	41

Of the patients invited to take the survey, 7% (n = 15) declined. Among the patients who agreed to be screened, 7% needed help using the iPad and 4% needed assistance with reading. The median time to complete the survey was 16 minutes (range 6-40 minutes). Responses to the final question of the survey revealed that 87% of the patients agreed it was a good idea for physicians to ask the behavioral health questions included in the survey.

In the patient population screened, 56% screened positive for anxiety, and 38% had scores consistent with moderate to severe depression. In addition, 34% screened positive for PTSD. The self-reported prevalence of an eating disorder, substance use disorder (SUD), and suicide ideation were much lower. The results are detailed in Figure 2.

Figure 2.

Patients screening positive for behavioral health problems (n = 196).

The consistency or agreement between the clinic diagnosis and BH-Works screening for depression was good (Kappa = 0.368437, P < .0001). Nonetheless, patients still were more likely to have a clinical diagnosis of depression entered into the EMR when using the BH-Works screening tool (17.3% vs 5.4%, OR = 3.682 [95% CI: 2.28-5.94], P < .001).

Of those patients who screened positive, 15% (n = 30) were already receiving some type of behavioral health service to address the issue, with 19% (n = 38) of additional patients referred to one of the collaborating behavioral health providers. In all, 37 of 38 patients referred had follow-up communication to assess whether linkage to services actually occurred; 27% (n = 10) of patients had made an appointment with or completed an appointment with a behavioral health provider.

Qualitative analysis on their decision-making process regarding follow-up care revealed 4 themes among the respondents: (1) acceptors and activated, (2) deniers and skeptics, (3) stigma impacted, and (4) social determinants interference (Table 2).

Table 2.

Themes of Patient Decision Making to Follow-up With Behavioral Health Referrals.

Themes	Descriptions	Quotes
Accepting and activated	Patients who accepted the need for behavioral health services and were in the process of or already engaged with a community agency.	“I attended one session and found it very helpful”“I have an appointment scheduled for next week”
Denying and skepticism	Patients who did not feel that they had a serious problem, or did not think behavioral health specialty care would be helpful.	“… my family says that I don’t need help”
Denying and skepticism		“I don’t see how they will help me”
Impacted by stigma	The role that stigma played in some patients’ decision making.	“… a patient told me that she would not seek help because if she did they might take her children away”
Impacted by stigma		“I don’t want anyone to know I have a mental problem”
Social determinants	Financial concerns and lack of transportation impacted decision making	“I haven’t gone because I don’t have a ride”
Social determinants		“I don’t have insurance”
		“I can’t afford any co-pays”

Use of the BH-Works tool in the practice setting required several elements of leadership and change management, superimposed on a framework of ongoing collaboration and performance improvement. Four themes were identified that require attention to achieve success: (1) sufficient resources to support the project, (2) leadership support, (3) managing the complexity of implementing change, and (4) patient-centeredness. Details of these themes are presented in Table 3.

Table 3.

Key Factors in Implementing the Behavioral Health Screening Tool.

Themes	Definitions	Descriptions	Quotes
Resources	Financial, human, and technical resources that affected the project’s success and sustainability	Facilitators: project management, regular team meetings, patient navigation via a CHW, financial support for the pilot project for software and tablets, community health alliance, data capture and reporting from the tool	“We need to add care management to the infrastructure”
		Barriers: no care management in the practice, limited financial reimbursement for the screening was a barrier to sustainability
Leadership Support	The role that leadership played in supporting the project in the practice	Facilitators: engagement and hands-on approach by some senior leaders, practice-level leaders were critical, decision making was at the practice level	“It was important to include the docs in planning”
		Barriers: strategic plan for further dissemination to other practices was not well developed	“We needed written expectations about the future beyond the project”
Complexity of Implementing the Innovation		Key features that impacted implementing a complex change in practice	“The workflows were very helpful”
		Facilitators: support from the BH-Works technical team, well-defined workflow processes, using PDCA cycles for continuous improvement, aligned with professional goals and recommendations from external professional organizations, and open communication between team members as part of the learning process	“Lack of integration into the EMR complicated the processes”
		Barriers: degree of adoption varied among practice staff, inability to make technical changes in the software program and EMR during a pilot project	“Front desk staff didn’t inculcate this into their everyday work”
			“The implementation process felt rushed”
			“This added value to our work”
			“This is a good thing for our patients’
			“Needed more decision making from the frontline workers”
Patient-centeredness		The role patient data and experience had on identifying gaps in organizational processes and team members’ acceptance of the tool	“. . . technology frustrated some patients”
		Facilitators: collecting patient process data and observing patient responses related to patient use of the screening tool and follow-up with referrals, PDCA cycles	“I had to help patients finish their screening in the waiting room”
		Barriers: absence of a patient in the design phase	“In the future we should involve patients in planning process”

Abbreviations: CHW, community health worker; EMR, electronic medical record; PDCA, Plan-Do-Check-Act.

Discussion

This study examined the outcomes of a primary care behavioral health quality improvement program in an academic urban setting that provides care for an economically and socially challenged population. The results reveal that the program was effective in screening for behavioral health problems and improving rates of patients diagnosed with depression. The majority of patients screened were positive about the survey. There was a higher than expected rate of moderate to severe depression, anxiety, and PTSD. Yet, only 27% of patients followed through with behavioral health referrals despite navigational assistance provided by a CHW and assured access to care through a community agency engaged with the Behavioral Health Alliance. Further qualitative analysis revealed that complex patient factors affected patient decision making regarding follow-up with behavioral health care. These included the following themes: accepting the diagnosis and activated to engage in care; denying the diagnosis and skepticism about providers; affected by the stigma of behavioral health diagnosis; and interference of social determinants of health.

As expected, use of a screening tool improved rates of patients diagnosed with depression. The lack of translation of a depression diagnosis from the behavioral health IT tool into the EMR likely was impeded by lack of electronic integration into the EMR. Other practices seeking to develop integrated models of care have met with similar challenges.^17-20 Integration of screening tools directly into the EMR likely would improve these parameters.

The results of 3 domains within the survey—depression, PTSD, and anxiety—were important practice findings in this study. The 38% prevalence of patients who screened positive for moderate to severe depression was considerably higher than national rates for major depression, which are between 3% and 5%.²¹ Present study rates are similar to matched patient cohorts experiencing significant poverty, as well as those with higher minority populations.^22,23 Not unexpectedly, a third of the patients screened positive for PTSD. Centralization of poverty in urban communities such as that in the present study is associated with high risks of exposure to traumatic events such as violence.²⁴ The trauma associated with this environment leaves indelible marks on the mental health of the community, including PTSD.²⁵ The significant number of patients who screened positive for anxiety may be a reflection of this observation. Of note, disease mechanism boundaries between depression and anxiety overlap are yet to be fully elucidated.²⁶ Overall, these findings emphasize the importance of screening for behavioral health issues and practice-level change to address behavioral health problems for vulnerable populations.

In the absence of behavioral health specialty care in the practice, the Behavioral Health Alliance was an integral project component whereby working relationships between primary and specialty care were formed to ensure timely access to and communication regarding care. Although the Alliance was helpful in establishing connections, staffing within the practice was important to bridge the organizations. Integrated care, by definition, is delivered in teams and therefore it is not surprising that the clinical team perceived the involvement of a dedicated CHW integral to facilitating connections to treatment. Even with additional staffing, barriers existed in linking patients to services.

Overall, African American patients have been shown to use mental health services at about one half the rate of white Americans, with Asian Americans at about one third the rate.²⁷ Present study results are similar to other findings that have shown that barriers such as stigma, skepticism, and social determinants perpetuate health inequities, particularly among underserved patients with regard to gaps in treatment.^28,29 For African Americans, Asians, and Latinos, mistrust of health care providers and concerns about cultural proficiency with their ethnic-racial group affects their comfort in talking to professionals.^28,29 Studies have had mixed results on the role of skepticism in African American patients, with some reporting that African Americans are more skeptical about treatment when compared to non-Hispanic whites.^30-32 Patient denial of a mental illness has been linked to the known stigma of these disorders and can be a coping mechanism that will interfere with acceptance of treatment.³³ Additionally, social determinants of health were important barriers, with financial concerns noted as the primary barrier reported in the literature.³⁴ These results show that additional studies of factors affecting follow-up with behavioral health care in racially, ethnically, and culturally diverse populations are needed. These additional studies should include patient activation measures, considering the qualitative data indicating that patients who accepted their diagnosis and were activated to seek help were those who followed through with their referrals. In the absence of studies on this topic, it is imperative that primary care providers and team members discuss patient readiness to engage in and potential barriers to behavioral health care.

Practice change requires attention to evidence-based attributes that have been shown to promote successful adoption. The literature has identified the following elements for primary care development including involvement with hospital leadership, community connections, space for a learning environment, system building, resources, and relationships, all of which were findings in this study.³⁵

There are certain limitations with the study design and feasibility of replicating the program. The study sample was smaller than anticipated because the office workflow was slowed by the survey process. Completion of a survey online prior to the visit through the EMR patient portal would be a more efficient method to conduct the survey. Other potential solutions might include adjustment of patient arrival time, changes in clinic workflow, or decreasing the length of the survey to concentrate on depression and PTSD. Many EMRs have built-in mechanisms utilizing evidence-based tools for screening. The added benefits of using this tool were the quality improvement reporting capabilities, support in documentation, and linkages to community-based services.

Another limitation included the design of the study as a performance improvement project, which resulted in the use of the physicians’ panels acting as an internal control group, which may have introduced bias. Expansion of this project to a formal research protocol will enhance the statistical accuracy.

Conclusion

The integration of mental health screening in the primary care setting utilizing an IT platform increased the diagnosis of depression, PTSD, and anxiety. Coupling this effort with physician referrals to a well-defined group of community agencies participating in a Behavioral Health Alliance was only partially successful despite the navigation skills of a CHW. Outcomes of this performance improvement design suggest that addressing mental health issues to improve the health of communities is a complex, multifactorial process that requires additional solutions and study.

Footnotes

Acknowledgements

Xiaoning Lu, Department of Clinical Sciences, Temple University, for his assistance with statistical analysis.

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Drs Swavely, O’Gurek, Schieber, Yu, and Freeman, and Ms Whyte declare that there are no conflicts of interest. Dr Tien is the chief scientific officer for mdlogix, the health care information technology company for BH-Works.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Deborah Swavely

References

Reeves

Strine

Pratt

, et al. Mental illness surveillance among adults in the United States. MMWR Supp. 2011;60(suppl 3):1-29.

Centers for Disease Control and Prevention. Workplace health promotion. https://www.cdc.gov/workplacehealthpromotion/index.html?CDC_AA_refVal=https%3A%2F%2Fwww.cdc.gov%2Fworkplacehealthpromotion%2Findex.htm. Accessed November 2018.

Chapel

Ritchen

Ahnang

Wang

Prevalence of medical costs of chronic disease among Medicare beneficiaries. Am J Prev Med. 2017;53(6 suppl 2):S147-S154.

American Psychiatric Association. Mental health disparities: diverse populations. https://www.psychiatry.org/psychiatrists/cultural-competency/mental-health-disparities. Accessed October 2018.

Cook

Trinh

Hou

Progovac

AM.

Trends in racial-ethnic disparities in access to mental health care, 2004-2012. Psychiatr Serv. 2016;68:9-16.

Memon

Taylor

Mohebati

, et al. Perceived barriers to accessing mental health services among black and minority ethnic (BME) communities: a qualitative study in Southeast England. BMJ Open. 2016;6:e012337. doi:10.1136/bmjopen-2016-012337

US Preventive Services Task Force. Final recommendation statement—Depression in adults: screening. https://www.uspreventiveservicestaskforce.org/Page/Document/RecommendationStatementFinal/depression-in-adults-screening1. Published January 2016. Accessed October 7, 2017.

Cunningham

PJ.

Beyond parity: primary care physicians’ perspectives on access to mental health care. Health Aff (Millwood). 2009;28:490-501.

American Academy of Family Physicians. Mental health care services by family physicians (Position Paper). https://www.aafp.org/about/policies/all/mental-services.html. Accessed October 2018.

10.

Temple University Hospital. 2016 Community health needs assessment. https://tuh.templehealth.org/upload/docs/TUHSPUBLIC/TUH-CHNA-2016.pdf. Accessed November 2018.

11.

Diamond

Levy

Bevans

, et al. Development, validation, and utility of internet-based, behavioral health screen for adolescents. Pediatrics. 2010;126:163-170.

12.

Greenhalgh

Robert

Bate

Macfarlane

Kyriakidou

Diffusion of Innovations in Health Services Organizations: A Systematic Literature Review. Malden, MA: Blackwell; 2005.

13.

Weick

Sutcliffe

Obstfeld

Organizing and the process of sensemaking. Organizational Science. 2005;(1694):409-421.

14.

Agency for Healthcare Research and Quality. Identify defects through sensemaking, facilitator notes. https://www.ahrq.gov/professionals/education/curriculum-tools/cusptoolkit/modules/identify/identifynotes.html. Published 2012. Accessed April 3, 2017.

15.

Brooks

McCluskey

Turley

King

The utility of template analysis in qualitative psychology research. Qual Res Psychol. 2015;12:202-222.

16.

Crabtree

Miller

, eds. Doing Qualitative Research. 2nd ed. Thousand Oaks, CA: Sage; 1999.

17.

Spear

Schmidhofer

Ambiguity and workarounds as contributors to medical error. Ann Intern Med. 2005;142:627-630.

18.

Saleem

Russ

Neddo

Blades

Doebbling

Foresman

BH.

Paper persistence, workarounds, and communication breakdowns in computerized consultation management. Int J Med Inform. 2011;80:466-479.

19.

Karsh

Holden

Alper

CKL

. A human factors engineering paradigm for patient safety: designing to support the performance of the healthcare professional. Qual Saf Health Care. 2006;15(suppl 1):i59-i65.

20.

Karsh

Clinical Practice Improvement and Redesign: How Change in Workflow Can Be Supported by Clinical Decision Support (AHRQ Publication No.09-0054-EF). Rockville, MD: Agency for Healthcare Research and Quality; 2009.

21.

Kessler

Chiu

Demler

Merikangas

Walters

EE.

Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the national comorbidity survey replication. Arch Gen Psychiatry. 2005;62:617-627.

22.

Katon

Depression research in under-resourced populations: an academic-community partnership. J Gen Intern Med. 2013;28:1255-1257.

23.

Santiago

Kaltman

Miranda

Poverty and mental health: how do low-income adults and children fare in psychotherapy?

J Clin Psychol. 2013;69:115-126.

24.

Horowitz

McKay

Marshall

Community violence and urban families: experiences, effects, and directions for intervention. Am J Orthopsychiatry. 2010;75:356-368.

25.

Overstreet

Braun

Exposure to community violence and post-traumatic stress symptoms: mediating factors. Am J Orthopsychiatry. 2000;70:263-271.

26.

Langer

Tonge

Piccirillo

Rodebaugh

Thomopson

Gotlib

Symptoms of social anxiety disorder and major depressive disorder: a network perspective. J Affect Disord. 2018;249:531-538.

27.

National Alliance on Mental Illness. Mental health by the numbers. http://www.nami.org/Learn-More/Mental-Health-By-the-Numbers. Published 2017. Accessed December 2018.

28.

Miranda

Ong

Jones

, et al. Community-partnered evaluation of depression services for clients of community-based agencies in under-resources communities in Los Angeles. J Gen Intern Med. 2013;28:1279-1287.

29.

Echeverry

JJ.

Treatment barriers accessing and accepting professional help. In: Garcia

Zea

, eds. Psychological Interventions and Research with Latino Populations. Needham Heights, MA: Allyn & Bacon; 1997.

30.

Hewins-Maroney

Schumaker

Williams

Health seeking behaviors of African Americans: implications for health administration. J Health Hum Serv Adm. 2005;28:68-95.

31.

Schnitter

Pescosoldeo

Groghan

Are African Americans really less willing to use healthcare?

Social Problems. 2005;52:255-271.

32.

Snowden

Barriers to effective mental health services for African Americans. Ment Health Serv Res. 2001;3:181-187.

33.

Mojkavai

Olfan

Sampson

, et al. Barriers to mental health treatment: results from the National Comorbidity Survey Replication. Psychol Med. 2011;41:1751-1761.

34.

Keith

La Vesit

. Social, economic, and health determinants of the use of health care services by Whites, African Americans, and Mexicans. In: Achieving Equitable Access: Studies of Health Care Issues Affecting Hispanics and African Americans, 1990;27:203-224.

35.

Miller

Crabtree

Nutting

Stange

Jaen

Primary care practice development: a patient-centered approach. Ann Fam Med. 2010;8(suppl):S68-S79.