Identifying Strategies for Strengthening Behavioral and Social Science Research Study Conduct

Abstract

Behavioral and social science research (BSSR) studies are less stringently regulated compared to drug and device studies. At our university, internal quality assurance activities revealed BSSR studies have more, but similar, issues compared to other studies. However, most institutional resources are tailored to support drug and device research, leaving an evidence gap regarding effective strategies for fostering rigorous BSSR study conduct. Two campus units partnered to undertake a two-phase study. First, we characterized compliance issues based on BSSR study audit data. Second, we conducted interviews with behavioral researchers and others to discuss contributors to compliance issues and potential mitigating strategies. Through thematic analysis, we identified problems at that can be addressed at individual, study team, and university systems levels. Study results outline potential strategies to reduce compliance issues.

Keywords

Behavioral social science research research ethics ethics education data safety monitoring behavioral research regulatory compliance translational science quality

Introduction

Behavioral and social science research (BSSR) is an integral part of translational science and is essential for finding ways to manage chronic health conditions (Dils et al., 2024; LeBlanc et al., 2018; Upsher et al., 2021). Data from a recent study showed that about half of new investigator-initiated trials at our institution from 2008–2014 were BSSR (Riley et al., 2020). Like drug and device studies, BSSR studies need to be conducted with rigor, adhering to the rules, policies, and standards of the regulatory bodies that govern them, such as Institutional Review Boards (IRB). However, when compliance issues arise, strategies to improve and prevent them may require a tailored approach for BSSR due to the inherent differences in this type of research compared to drug and device studies. BSSR studies tend to involve minimal risk and are not regulated by the Food and Drug Administration (FDA) like drug and device studies. While the FDA has stringent regulations involving study adverse event reporting and monitoring, which provide some additional safeguards for these investigators, BSSR studies are funded by various sponsors that may have differing reporting requirements or are independently funded. Despite an equal need across all types of research to have strong practices in reporting events and conducting research, some members of the BSSR workforce may not be aware of how best to conduct their studies with rigor given they come from various disciplines who may have variable training. Finally, additional complexities of BSSR may increase the challenges of upholding study rigor compared to drug and device studies, such as needing to train and monitor study interventionists to uphold fidelity, conducting studies in less controlled settings such as the community, and collecting and managing data in the field from wearables or study apps.

No literature could be found that exclusively examined compliance issues, such as issues with the informed consent process, adverse event reporting, and protocol adherence in BSSR, although some studies have examined compliance in human participants research as a whole. In a study of academic centers (including our university) that perform routine audits for regulatory compliance in IRB-approved clinical research studies, most issues that were identified involved problems with the informed consent form or process, while others involved inconsistencies between the study protocol being used and the one approved by the IRB; study documents not being stored as indicated on the protocol; and failure to report events or protocol deviations (Cox et al., 2023). In a study using retrospective data to examine compliance throughout conduct of 193 studies, some similar issues were found, such as lack of consistency between protocol and study conduct, i.e., performing analyses that were not consistent with what was stated in the study objective, and failure to report protocol deviations (Gajbhiye et al., 2020). Although these studies involve human participants, which include BSSR studies, we are limited in knowing if and how these findings translate to BSSR studies and therefore highlight the need for this work. A better understanding of these determinants for BSSR is needed to develop strategies to mitigate BSSR study conduct issues.

The goals of this study were to characterize the compliance issues observed during post-approval reviews of BSSR studies (phase 1) and to further understand results from the perspectives of researchers, research staff, research compliance staff, and IRB administrators in phase 2 in order to suggest some potential solutions to improve BSSR study rigor.

Methods

The study was conducted by our university's quality assurance unit (QA Unit) and a program dedicated to supporting behavioral research on campus. The QA Unit conducts reviews of studies in which human participants participate across schools and colleges within the university and tracks the types and prevalence of compliance issues. The behavioral research program was engaged to help support potential strategies that might mitigate identified issues with the university BSSR study conduct. This study was supported by a pilot award through an NIH Clinical Translational Award grant at our university, and the research was determined to be not regulated through the university's medical school IRB meaning the study does not involve human participants research that requires oversight of the IRB.

We used an explanatory sequential mixed methods design (Creswell & Plano Clark, 2018), beginning with a quantitative examination of compliance issues. We then used the results to develop a qualitative interview guide and identify a purposive sample with the intent of further explaining the quantitative results.

Phase 1 Methods

In phase 1, we characterized compliance issues using the QA unit's data. The purpose of their routine reviews is to assist researchers and research staff by reviewing institutional, state, and federal policies and best practices that apply to the study; evaluate study conduct; and ensure proper record-keeping of all study-related documents. Generally, reviews consist of an interview with the research team and a review of records. Each review is tailored to the nature and scope of the study and may include, but is not limited to, a systematic review of: study team roles and responsibilities, recruitment procedures, screening and eligibility determination process, consent process and documentation, implementation of various study procedures, study document management and record-keeping, data safety and monitoring, IRB reporting requirements, data confidentiality procedures, principal investigator (PI) oversight, and study team training. Reviews are summarized in a report outlining factual observations and associated recommended corrective actions. In addition to study-level reports, routine review findings are tracked in a database to analyze trends over time and across studies. In the QA unit's database, issues are classified using an established list of previously identified issues and are grouped into topics that correspond to the above areas of review.

Using the existing database, our team searched for BSSR studies that underwent reviews from 2017–2023. Studies were identified through a search of generated keywords by behavioral research program investigators within a university research institute and a review of study titles. The investigators met to generate as complete of list as possible of keywords related to behavioral intervention studies and other terms related to human behavior (e.g., physical activity, sleep, exercise, mindfulness, caregiver, self-management). The behavioral research program investigators were blinded to the research investigators who conducted these studies. From the subset of BSSR studies, we generated reports of the types and prevalence of compliance issues. We examined differences in issues based on characteristics such as the academic rank of the principal investigator (PI), grant funding type, and study risk level using one-way analyses of variance and study type using t-tests.

Phase 2 Methods

In phase 2, using a qualitative descriptive design, we sought to further explain and describe problems identified during phase 1 and to identify potential solutions. The results of phase 1 informed both sampling and interview questions in phase 2. Through the connecting integration process (Fetters et al., 2013), individuals for the qualitative interviews were purposively sampled using a maximal variation strategy to seek breadth in terms of school or college, rank, and student involvement based on the same range of characteristics examined in phase 1. Individuals were selected irrespective of studies or personnel in phase 1 as the studies that are presented in phase 1 were de-identified to the study team. The sample size was guided by the concepts of information power—considering the number needed based on the complexity and no planned comparative qualitative analysis (Malterud et al., 2015) and data saturation, which is a point at which continuing to collect data is not adding complexity to findings (Guest et al., 2006).

We conducted semi-structured interviews with researchers, research staff, and IRB or research compliance staff. A semi-structured interview protocol began with a series of elicitation questions based on issues found in phase 1 through a process called building integration (Fetters et al., 2013) in which we mapped major issues identified in phase 1 to specific questions and to construct realistic vignettes. The first set of questions included tables and graphics to outline the prevalence and type of common compliance issues with prompts to discuss what could have led to the major categories of compliance issues and what might prevent them. We then presented hypothetical vignettes followed by similar questions about potential causes and solutions which were used to generate qualitative themes (see Supplemental Material for the full interview guide). The interview concluded by asking about training needs and recommendations to mitigate compliance issues. All interviews were conducted by a study investigator experienced with qualitative methods and lasted 30–40 minutes and occurred from November 2023-January 2024. Interviews were audio recorded and transcribed.

We imported all transcripts into MAXQDA, a qualitative software program for managing and analyzing data. We used a thematic analysis approach to understand perceived barriers and potential solutions to identified issues (Braun & Clarke, 2023). After reviewing transcripts, we developed a codebook to apply to the transcripts and refined it throughout the coding process. Two study team members conducted coding and discussed coded transcripts through a consensus process to ensure agreement. Through MAXQDA queries, we identified initial themes based on patterns and clusters among codes and then met as a research team to discuss and refine themes.

Phase 1 Quantitative Results

The QA unit searched their database from 2017–2023, which contained 172 records, four of which were incomplete. The remaining 168 studies were reviewed by the unit and the study team; 52 were deemed eligible and were included in this study. Forty-four studies (85%) were classified as no more than minimal risk, 7 were minor increase over minimal risk (14%), and one was moderate risk. From the regulatory perspective, very few (n = 5, < 10%) had major research compliance findings, meaning findings that can potentially rise to the level of serious and/or continuing noncompliance. Most studies (n = 37, 72%) had issues which would be reportable to the IRB, meaning minor deviations that do not pose risk to participants, such as minor protocol or consent deviations, missed documentation, or training delays. Studies had an average of 5.2 compliance issues (range 1–11, SD = 2.4).

Table 1 shows the differences in the number of compliance issues by the PI and study characteristics. There were no significant differences in the number of issues based on any of the characteristics measured. Table 2 summarizes the most frequently observed issues in the graphic and, as will be discussed in the phase 2 findings, maps these to the qualitative findings that emerged from interviews where participants were asked to comment on their thoughts of these presented compliance issues. The most frequent type of issue occurred in findings of the informed consent process (95 issues within and across studies), such as using outdated informed consent forms, and not using the “options” sections in the informed consent forms consistently or accurately (e.g., consent for sub-studies, use of data for future research). There were 71 issues related to IRB submission, for example, needing to update study team members listed in the application. Twenty-four issues with proper recordkeeping were also found, such as study documentation inappropriately corrected (e.g., errors on data collection forms corrected by obscuring incorrect data and writing new data without initials or date). Twenty-three issues with protocol adherence were found. These issues included accidental deviation or variance from protocol and alterations to protocol made without IRB approval. Twenty issues with participant eligibility included source documentation to verify eligibility not being consistently on file, study documents lacking an eligibility form or checklist; or lack of documentation that exceptions to eligibility criteria were submitted to the IRB as examples. Regarding training compliance issues (of which there were 13), most concerned not having good clinical practice (GCP) training for all study staff on file. Other issues included lack of other training documentation. Ten issues with data use agreements pertained to sharing data prior to obtaining a fully executed agreement.

Table 1.

Differences in Prevalence of Issues by PI and Study Characteristics Across 52 Studies.

Characteristic	N	Number of Issues	P value
PI rank			0.79
Assistant Professor	21	5.5 (2.3)
Associate Professor	13	4.7 (2.8)
Professor	13	5.1 (1.7)
Other	5	5.0 (3.3)
Study type			0.34
Clinical Trial	20	5.6 (2.5)
Not Clinical trial	32	4.9 (2.2)
Funding type			0.82
Federal	27	5.3 (2.4)
Non-federal	21	4.9 (2.2)
None	4	5.3 (3.2)
Study risk level			0.63
No more than minimal risk	44	5.1 (2.4)
Minor increase over minimal	7	5.9 (2.1)
Moderate risk	1	4.0 (0)

Note. One way ANOVAs were used to examine between group differences in PI rank, funding type, and study risk level by number of regulatory issues. for PI rank, F (3,48) = .35; For funding type F (2,49) = .20; and study risk level F (2,49) = .46. A t-test was used to examine study type by number of regulatory issues: t = .97 (mean difference = .66; 95% CI = −.69–1.98).

Table 2.

Mixed Methods Joint Display of Most Prevalent Compliance Issues, Qualitative Results, and Mixed Methods Interpretation.

Quantitative Results Frequency of Compliance Issues by Type Across 52 BSSR Studies	Qualitative Results Interview Themes	Mixed Methods Interpretation
	Potential Sources of Issues The need for understanding roles and requirements for study conduct The importance of developing teams and providing professional development The need for revising IRB processes to facilitate compliance Potential Solutions Providing tailored training for staff and investigators Providing research team support including checklists and consultation Improving mechanisms to communicate with the IRB Developing communities of practice	Problems at the level of individual staff and researcher, research teams, and the IRB system level create room for error and can exacerbate all types of compliance issues.Solutions can address all compliance issues though some are simpler and less costly to implement than others. Access to training material, providing readily accessible checklists, and advertising available consultation likely require minimal resources. Developing better communication tools and communities of practice are more intensive but could lead to change at the system level.

Phase 2 Qualitative Findings

Of 34 individuals invited for an interview through an email invitation, 17 participated. The characteristics of this sample are reported in Table 3. We generated seven themes (summarized in Table 2), which we organized into those at the individual, team/relationship, and system levels (Supplemental Tables 1 and 2). The themes describing what led to study conduct problems were: the need for understanding roles and requirements for study conduct, the importance of developing teams and providing professional development, and the need for revising IRB processes to facilitate compliance (Supplemental Table 1). We also generated themes based on suggested solutions: providing tailored training for staff and investigators, providing research team support including checklists and consultation, improving mechanisms to communicate with the IRB, and developing communities of practice (Supplemental Table 2).

Table 3.

Interviewee Characteristics (N = 17).

Characteristics
Age (mean (SD))	43.6 (9.2)
Race
White Asian	16 (94.1%)1 (5.9%)
Ethnicity
Hispanic	17 (100.0%)
Non-Hispanic	0 (0.0%)
Highest level of school completed
Bachelor's degree	4 (23.5%)
Master's degree	3 (17.6%)
Doctoral degree	10 (58.8%)
Primary role at the university
Administrator	1 (5.9%)
Research staff	8 (47.1%)
Faculty – research	3 (17.6%)
Faculty – instructional/tenure	5 (29.4%)
Department, Division, or Unit
Family Medicine	3 (17.6%)
Anesthesiology	2 (11.8%)
Internal Medicine	2 (11.8%)
Psychiatry	2 (11.8%)
School of Public Health	2 (11.8%)
Physical Medicine & Rehabilitation	1 (5.9%)
School of Nursing	1 (5.9%)
School of Social Work	1 (5.9%)
Clinical Trials Support Office	1 (5.9%)
IRB	1 (5.9%)
Quality Assurance Unit	1 (5.9%)
Years in research (mean (SD))	17.6 (6.7)

Sources of Research Compliance Issues

The Need for Understanding Roles and Requirements for Study Conduct

At the individual level, participants explained that a lack of awareness of the importance of requirements at the individual level leads to study compliance issues. They explained that some investigators and staff seem unaware of requirements (e.g., reporting change in study staff or protocol deviations) or do not understand the importance of requirements. A participant explained,

“It's just unawareness. Not being aware of the importance of some of these things…I just don’t think people pay attention to some of those issues largely because we don’t have to report everything to the FDA and good luck thank you that we don’t.” (P1, Research staff)

In addition to lack of awareness of regularly issues, many described problems with research protocols. For example, investigators may struggle to translate a grant application into a study protocol, often reusing a grant narrative for the protocol. While similarities may exist, grants do not typically contain the detailed standard operating procedures needed for a protocol:

“I think we all struggle with doing the IRB application after a grant is funded as an example. And then trying to fit your grant into a protocol which aren't always going to be exactly the same.” (P14, Researcher)

Moreover, protocols are often not sufficiently flexible. They often do not anticipate deviations and include contingencies. When deviations occur, amendments and reporting to the IRB is needed. These studies are more vulnerable to study compliance issues (e.g., not reporting deviations) than studies following flexible, approved protocols:

“So I guess the idea there is maybe like more guidance and examples of ways to make the protocol to kind of anticipate things that you don't necessarily want to be a deviation. So you're making it a more flexible protocol in a way that doesn't compromise the research, but in a way that saves unnecessary like reporting and stuff. (P03, Researcher)

The Importance of Developing Teams and Providing Professional Development

At the level of teams and relationships, problems may stem from a lack of clarity about the scope of study team member roles. Regarding staff, many described inadequate professional development for project managers, coordinators, and research assistants. Some teams experience high turnover, which amplifies challenges with onboarding training and ongoing skill development. Mentoring and professional development is needed across all levels of the team:

“… that is ongoing training and mentorship of project managers who are the ones who are in the trenches working with these low-level research assistants. How do we support them? There's a lot of training targeted at the research assistant. There's not a lot of training targeted at the project manager, the people who oversee the folks who are doing the grunt work. How do we help them?” (P16, Research staff)

The responsibility extends to principal investigators, who have a major role in developing their team and setting expectations. As explained by participants, some principal investigators are ill-prepared:

“But again, one of the things that I say a lot about PIs is that they are researchers. They're not managers. They just happen to be thrust into the role of being a manager. They also are researchers, but they're not regulatory or compliance-focused. They're data focused. They're writing the next paper. They're tenure focused. So those are those are their priorities.” (P16, Research staff)

A related phenomenon is the “absent PI,” a term some used to describe investigators who lack availability to their staff and are perhaps handling too many projects and responsibilities. An investigator who is not engaged or available to their team could lead to more compliance issues:

“I also think honestly, absent PIs. So I’ve had the privilege of working with some amazing PIs, I’ve had the misfortune of working with some less than amazing PIs and a key feature of an amazing PI is someone who is accessible and responsive…And if you have a team of 1 person, 1 study staff and you have a PI who is not accessible or not responsive to read through these things and go over them, who is going to catch the mistake?” (P1, Research staff)

The Need for Revising IRB Processes to Facilitate Compliance

Other problems stem from the IRB system level. A common problem cited was the IRB submission system, described as convoluted, repetitive, and difficult to navigate. Staff and investigators explained system limitations that can promulgate compliance issues:

“Due to limitations of the system, there's multiple opportunities for study teams to be inconsistent. There are a number of items on the application that get similar pieces of information” (P8, IRB Member)

When the same information needs to be entered in multiple places, it opens the door for inconsistencies within the application and supporting study documents (e.g., consent forms). Staff and investigators universally expressed frustration with the application process. In addition, staff and investigators explained variability among IRB staff reviewing applications.

Providing Tailored Training for Staff and Researchers

Training was a commonly-cited need to reduce study compliance issues. Most of the suggestions were to add relevant continued training for study teams. Some suggested having training tailored for investigators or study staff to more clearly address the different needs of each. Also, participants frequently suggested making training material easily accessible after completion. They noted having material (e.g., slides and videos) would be helpful to refer to when preparing IRB application and amendments and when conducting the study after approved. A research team staff member explained the suggestion for providing resources:

“I think having useful quick little SOPs [standard operating procedures] or something for them to check for the pieces that they're required to do, like submitting an amendment or an application or something through clinicaltrials.gov where they have to approve and release the record, I think having those resources available that they can look through really quickly are super helpful.” (P24, Research staff)

Providing Research Team Support Including Consultation and Checklists

Many suggested providing resources for teams to ensure consistency and learn collectively. At the initiation of a study, a few participants described the helpfulness of a study startup consultation provided to the university community. These consultations are offered at no cost, yet it seemed most were unaware of the service. Advertising and disseminating the service may help. Study start-up consultations are offered by the QA team for newly approved studies, after IRB approval and before enrollment of participants. The purpose of these consultations is to assess the study team's preparedness to implement the IRB approved protocol, offer suggestions, and establish a forum of communication between the QA team and study team members. Specifically, during a study start-up consult, the QA team provides tools and resources to support the conduct of the study, identifies practices related to the consenting process and documentation, and share tips and lessons learned from QA audits.

Some staff recommend checklists that they have used in the past or internally developed for their team. These checklists are not necessarily team-specific and could be helpful to other researchers and teams.

“So behavioral studies I've been part of in the past have used a checklist.” (P37, Researcher)

Checklists are helpful for self-assessment, particularly when used regularly, to help reduce compliance issues. Making checklists broadly available is likely easy to implement through a website.

Improving Mechanisms to Communicate with the IRB

Two primary solutions focused on communication with the IRB: developing a user-friendly application and enhancing communication methods with the IRB. First, many suggestions focused on implementing a protocol-based IRB application that is more flexible and anticipates contingencies. A well-designed protocol will require fewer amendments and lead to fewer reportable events. Technology may also be helpful to check for inconsistencies within the IRB application.

Other suggestions focused on how to obtain information and communicate with the IRB. Participants suggested having regularly scheduled office hours for drop-in consultation, identifying liaisons who attend departmental meetings, offering a real-time chat function through the website. An investigator gave an example of having a communication mechanism for quick answers:

“There are so many times that questions will come up, or my colleagues will ask me questions because the process of submitting an amendment and waiting on that to go, and all that is really arduous and just having maybe even a less formal forum whereby you can contact an IRB administrator. Whether it's like a regularly monitored G-chat, or something like that would be really, really helpful.” (P9, Researcher)

These solutions focused on accessibility to the IRB for questions.

Developing Communities of Practice

Like the suggestion for better access to the IRB for questions, participants suggested having access to other faculty and staff expertise through a community of practice. The community of practice would consist of a forum for asking questions and sharing ideas across departments and other units. In addition, these communities can break down silos that often occur:

“When I talk to study team members like research assistants and coordinators and such so many of them across the research enterprise, whether they’re on the campus side or medical school side say they feel isolated and alone. If you were isolated and you have a question and you don’t know where to go, all of these things that you just showed they’re going to happen… I think a way of connecting people and resources so, hey here's an example of a protocol I wrote, here's a good way of talking about how we’re going to document something, or here's I use REDCap to document this, here's my data dictionary kind of thing.” (P1, Research staff)

A forum could be a developed through existing institutional communication channels, such as Slack, Teams, Tello, or other tools. These tools can facilitate the exchange of ideas and the need for ongoing guidance.

Integrating the Phase 1 Quantitative and Phase 2 Qualitative Results

Although the interview elicitation questions were designed to probe specific compliance issues identified in phase 1, the suggested reasons (i.e., sources of problems) were explained as root causes that could lead to multiple compliance issues. For example, the need to understand roles and requirements was implicated in informed consent issues, IRB submission inconsistencies, recordkeeping, and protocol adherence. Similarly, the themes regarding solutions also spanned multiple compliance issues. All potential solutions—training, checklists, communication, and communities of practice—could mitigate the most prevalent issues identified in phase 1 (see Table 2).

Discussion

We characterized compliance issues in BSSR studies at our institution and used the data to both identify sources of the problems and generate potential strategies to strengthen BSSR rigor. Although we exclusively sampled from BSSR studies, the compliance issues found were similar to those found in other studies that included data from both BSSR and drug and device studies (Cox et al., 2023; Gajbhiye et al., 2020). To help mitigate compliance issues in drug and device studies, investigators of drug and device studies are required to take training in good clinical practice that focuses on competencies needed for proper study conduct. This proactive strategy could also potentially help prevent compliance issues in BSSR, and a training specific to BSSR research best practices was developed to be equivalent to good clinical practice training (Murphy et al., 2018; Samuels et al., 2024). However, this training is typically only required for studies deemed BSSR clinical trials so may not be used broadly for the workforce of BSSR investigators.

Regarding potential strategies to reduce compliance, our participants suggested providing tailored training for staff and investigators, providing support for research teams including checklists and consultation, improving mechanisms to communicate with the IRB, and developing communities of practice. A multilevel approach to improving research integrity, such as regulatory compliance issues, has been recommended in the literature (Roje et al., 2023), and other articles have mentioned the need to impact the institutional culture which extends beyond just the individual researcher or team (Cox et al., 2023; DuBois & Antes, 2018; Geller et al., 2010; Lee, 2023). Regarding strategies to improve compliance issues, Cox and colleagues (2023) advocate for routine audits via post-approval monitoring programs, not as a punitive system but as a proactive strategy, to support investigators and teams with what is needed, such as training to ensure compliance and preserve research integrity. This process is what our program tries to emulate. This positive framing to ensure compliance may be particularly important to address barriers others have identified such as a lack of open communication due to power imbalances (Geller et al., 2010). Like our findings, training has been identified as a strategy to reduce compliance issues (Cox et al., 2023; DuBois & Antes, 2018; Roje et al., 2023). In one study, the authors proposed a model to improve research integrity at institutions (DuBois & Antes, 2018) and discussed five domains of which compliance was one aspect (the others being ethics, rigor and reproducibility, social value, and workplace relationships). They proposed that strategies to improve each domain would need to identify who would champion the effort, the interventions needed, and the ideal participants. When focused on compliance, they recommended champions of compliance officers and trained research staff to provide training to researchers and staff, whether through education modules, learning, or consultation.

Our study's specific solutions were generated with input from participants involved in BSSR The mixed methods results showed that compliance issues can be amplified from errors that happen at every level - individual staff and researcher, research teams, and the IRB system. However, this provides the opportunity to identify solutions that may be cross-cutting in their effect to address compliance issues and strengthen BSSR study conduct. It is important to note that the solutions require varying degrees of resources. Providing checklists and access to training material are relatively simple and could have an immediate benefit to individuals and teams. Solutions such as redesigning the IRB application processes, adding mechanisms to communicate with the IRB, and developing communities of practice are more resource intensive and may take more time to implement. However, they also have the potential for system-level changes that may mitigate compliance issues.

In conclusion, this study offers practical solutions to common compliance issues in BSSR studies. While the data sampled on BSSR studies were only taken from one university, future research could involve multiple institutions to determine if issues and generated strategies from this study are similar and generalizable.

Best Practices

The workforce of research investigators, staff, and students who conduct behavioral research should be well-supported by institutions that have approved their human participants research. Because behavioral research is unique in some ways compared to drug and device studies, understanding compliance issues specific to behavioral research studies is necessary. Many resources to help investigators conducting clinical trials are drug and device centric, which are not well-suited to teach the skills required for rigorous behavioral research study conduct. Providing the appropriate training, resources, and support should be a priority to support behavioral research rigor.

Research Agenda

Given the unique features of BSSR compared to drug, device, and biologic studies and the lack of research in this area, there is a critical need to further characterize BSSR compliance issues and develop solutions with input from those designing, conducting, and/or supporting BSSR. Our study found that solutions may be targeted at the individual, team, or systems levels. After enacting solutions at institutions, it will be important to examine whether implementation of particular strategies is having the intended effects with the workforce and on research quality.

Educational Implications

The potential solutions to compliance issues that were proposed in this study with input from our participants will be translated into strategies to increase research rigor. For instance, training opportunities, in-the-moment responses to questions, and targeted information in the IRB online portal are some examples of how the workforce will benefit from this work. The approach taken in this study, that combined quantitative data and qualitative feedback from participants to identify issues and create potential solutions, could serve as a model for other institutions that want to strengthen rigor in BSSR studies.

Supplemental Material

sj-docx-1-jre-10.1177_15562646241302396 - Supplemental material for Identifying Strategies for Strengthening Behavioral and Social Science Research Study Conduct

Supplemental material, sj-docx-1-jre-10.1177_15562646241302396 for Identifying Strategies for Strengthening Behavioral and Social Science Research Study Conduct by Susan L. Murphy, Timothy C. Guetterman, Elizabeth K. Haro and Sana Shakour in Journal of Empirical Research on Human Research Ethics

Footnotes

Acknowledgements

We thank Malak Ismail, MPH, and Gina Jay, PhD, for helping to compile and organize the data and supporting the research process.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the The Michigan Institute of Clinical and Health Research (MICHR), (grant number UM1TR004404).

ORCID iDs

Susan L. Murphy

Timothy C. Guetterman

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Susan L. Murphy, ScD, OTR, is a Professor in the Department of Physical Medicine and Rehabilitation and in the Rheumatology Division at the University of Michigan. She is also Director of the Behavioral Research Innovation and Support Program (BRISP) at the Michigan Institute for Clinical and Health Research. Dr. Murphy co-designed this project, conducted quantitative data analysis, worked with the team on data interpretation, and led manuscript writing.

Timothy C. Guetterman, PhD, MA, is an Assistant Professor in the Department of Family Medicine at the University of Michigan. He is also the Associate Director of the Michigan Mixed Methods Program at the University of Michigan. He designed and performed the mixed methods analysis for the study, conducted interviews, led the data interpretation process, and participated in manuscript writing.

Elizabeth K. Haro, MPH, is a research coordinator in the Department of Physical Medicine and Rehabilitation at the University of Michigan. She was responsible for the day-to-day project coordination, assisted with the qualitative analysis of the interviews, and participated in drafting the manuscript.

Sana Shakour, PhD, is the Director, Office of Research Compliance Administration, University of California, Santa Cruz and formerly Associate Director of the Human Research Protections Program (HRPP) at the University of Michigan. During the time of this study, she was responsible for the day-to-day operations, strategic planning, and collaboration with the University of Michigan IRBs and other HRPP component units, and oversaw the Office of Research Compliance Review. In her role, she co-designed this project, oversaw the data management and analysis of the behavioral research studies that were part of the routine audits sample included in this paper. She also participated fully in the analysis interpretation and manuscript writing.

References

Braun

Clarke

(2023). Thematic analysis. In APA Handbook of research methods in psychology: Research designs: Quantitative, qualitative, neuropsychological, and biological, Vol. 2, 2nd ed. (pp. 65–81). American Psychological Association. https://doi.org/10.1037/0000319-004

Cox

Solbakk

J. H.

Luthardt

Bernabe

R. D. L. C.

(2023). Institutional review boards and post-approval monitoring (PAM) of human research: Content analysis of select university (academic health center) web pages across the USA [article]. Current Medical Research and Opinion, 39(3), 341–350. https://doi.org/10.1080/03007995.2023.2175999

Creswell

J. W.

Plano Clark

V. L.

(2018). Designing and conducting mixed methods research (3rd ed.). Sage.

Dils

A. T.

O'Keefe

Dakka

Azar

Chen

Zhang

(2024). The efficacy of cognitive behavioral therapy for mental health and quality of life among individuals diagnosed with cancer: A systematic review and meta-analysis [Review]. Cancer Medicine, 13(16), Article e70063. https://doi.org/10.1002/cam4.70063

DuBois

J. M.

Antes

A. L.

(2018). Five dimensions of research ethics: A stakeholder framework for creating a climate of research integrity. Academic Medicine, 93(4), 550–555. https://doi.org/10.1097/acm.0000000000001966

Fetters

M. D.

Curry

L. A.

Creswell

J. W.

(2013). Achieving integration in mixed methods designs - principles and practices. Health Services Research, 48(6 PART2), 2134–2156. https://doi.org/10.1111/1475-6773.12117

Gajbhiye

S. V.

Jalgaonkar

S. V.

Dabba

S. G.

Surve

Lad

M. S.

(2020). Assessing completion reports for compliance with institutional ethics committee-approved protocols: An observational study. Indian Journal of Medical Ethics, V(2), 119–123. https://doi.org/10.20529/ijme.2020.025

Geller

Boyce

Ford

D. E.

Sugarman

(2010). Beyond “compliance": The role of institutional culture in promoting research integrity. Academic Medicine, 85(8), 1296–1302. https://doi.org/10.1097/ACM.0b013e3181e5f0e5

Guest

Bunce

Johnson

(2006). How many interviews are enough : An experiment with data saturation and variability. Field Methods, 18(1), 59–82. https://doi.org/10.1177/1525822x05279903

10.

LeBlanc

E. S.

Patnode

C. D.

Webber

E. M.

Redmond

Rushkin

O’Connor

E. A.

(2018). Behavioral and pharmacotherapy weight loss interventions to prevent obesity-related morbidity and mortality in adults: Updated evidence report and systematic review for the US preventive services task force. JAMA, 320(11), 1172–1191. https://doi.org/10.1001/jama.2018.7777

11.

Lee

L. M.

(2023). Research integrity and the regulatory-industrial complex. Accountability in Research, 31(7), 887–897. https://doi.org/10.1080/08989621.2023.2179395

12.

Malterud

Siersma

Guassora

(2015). Sample size in qualitative interview studies: Guided by information power. Qualitative Health Research, 26(13), 1753–1760. https://doi.org/10.1177/1049732315617444

13.

Murphy

S. L.

Samuels

Kolb

Behar-Horenstein

Champagne

Byks-Jazayeri

Hahn

Roth

M.-T.

Ennever

Bajwa

Singh

Homish

G. G.

Dubocovich

M. L.

(2018). Best practices in social and behavioral research: A multisite pilot evaluation of the good clinical practice online training course. Journal of Clinical and Translational Science, 2(2), 95–102. https://doi.org/10.1017/cts.2018.27 .

14.

Riley

W. T.

Bibb

Hargrave

Fearon

(2020). Publication rates from biomedical and behavioral and social science R01s funded by the National Institutes of Health [Article]. PloS one, 15(11 November), Article e0242271. https://doi.org/10.1371/journal.pone.0242271

15.

Roje

Reyes Elizondo

Kaltenbrunner

Buljan

Marušić

(2023). Factors influencing the promotion and implementation of research integrity in research performing and research funding organizations: A scoping review. Accountability in Research, 30(8), 633–671. https://doi.org/10.1080/08989621.2022.2073819

16.

Samuels

Janevic

M. R.

Harper

A. E.

Lyden

A. K.

Jay

G. M.

Champagne

Murphy

S. L.

(2024). Updating and evaluating a research best practices training course for social and behavioral research professionals [Article]. Journal of Clinical and Translational Science, 8(1), Article e12. https://doi.org/10.1017/cts.2023.702

17.

Upsher

Onabajo

Stahl

Ismail

Winkley

(2021). The effectiveness of behavior change techniques underpinning psychological interventions to improve glycemic levels for adults with type 2 diabetes: a meta-analysis [Systematic Review]. Frontiers in Clinical Diabetes and Healthcare, 2, 1–18. https://doi.org/10.3389/fcdhc.2021.699038

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

2.00 MB

Identifying Strategies for Strengthening Behavioral and Social Science Research Study Conduct

Abstract

Keywords

Introduction

Methods

Phase 1 Methods

Phase 2 Methods

Phase 1 Quantitative Results

Phase 2 Qualitative Findings

Sources of Research Compliance Issues

The Need for Understanding Roles and Requirements for Study Conduct

The Importance of Developing Teams and Providing Professional Development

The Need for Revising IRB Processes to Facilitate Compliance

Suggested Potential Solutions to Research Compliance Issues

Providing Tailored Training for Staff and Researchers

Providing Research Team Support Including Consultation and Checklists

Improving Mechanisms to Communicate with the IRB

Developing Communities of Practice

Integrating the Phase 1 Quantitative and Phase 2 Qualitative Results

Discussion

Best Practices

Research Agenda

Educational Implications

Supplemental Material

sj-docx-1-jre-10.1177_15562646241302396 - Supplemental material for Identifying Strategies for Strengthening Behavioral and Social Science Research Study Conduct

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

ORCID iDs

Supplemental Material

Author Biographies

References

Supplementary Material