Abstract
Introduction
In population health research, recruitment and retention of diverse populations is a cornerstone of high-quality studies. It is well recognized that improving participation of underrepresented populations in clinical and population health research will help increase the generalizability of the study findings, and therefore contribute to the improvement of health equity, foster greater trust in health care, and reduce health disparities (Oh et al., 2015; Rivas-Drake et al., 2016; Shabu et al., 2022; Stronks et al., 2013). However, the research community has struggled to successfully recruit and retain racial and ethnic minority participants (Aguirre et al., 2018). Of note, the Hispanic/Latino (henceforth referred to as Hispanic in this manuscript) population is among the fastest growing minority populations in the United States (US) and is projected to make up 29% of the total US population by 2060 (Colby & Ortman, 2014). However, Hispanics, especially older Hispanics, were underrepresented in both observational and interventional studies compared with non-Hispanic Whites using traditional recruiting strategies (Areán & Gallagher-Thompson, 1996; Gallagher-Thompson et al., 2004; Godden et al., 2010). Successful recruitment and retention of Hispanic participants may require targeted and novel approaches to compensate for historic recruiting shortfalls (Together We Make the Difference: National Strategy for Recruitment and Participation in Alzheimer's and Related Dementias Clinical Research, 2018).
The importance of Hispanic recruitment and participation in age-related cognition research is underscored by studies suggesting that Hispanic populations are at elevated risk for dementia compared to non-Hispanic Whites, and that cognitive outcomes vary by Hispanic subgroups (Garcia et al., 2019; Weden et al., 2017). Hispanics in the US suffer the disproportionate burden of risk factors for cognitive disorders such as socioeconomic status, educational attainment, having a sedentary lifestyle, and language barriers limiting access to healthcare and healthcare recommendations (Martinez-Miller et al., 2020; O'Bryant et al., 2008; Quiroz et al., 2022). Previous studies have suggested that recruitment success in aging and minority populations may require targeting socially isolated communities with technology-based recruitment strategies, building trust with potential participants, providing incentives for participation, using culturally appropriate recruitment strategies and materials, and engaging stakeholders to address barriers to research participation (Aragones et al., 2014; Gao et al., 2015; Lacey et al., 2017; Marshall et al., 2020; Napoles & Chadiha, 2011; Raman et al., 2021; Salazar et al., 2020). A 2021 review of facilitators and challenges to participation in Alzheimer's and dementia research among Hispanic individuals cited a lack of culturally appropriate resources and staffing, as well as too few studies involving Hispanic communities (Massett et al., 2021).
The COVID-19 pandemic disproportionately burdened Hispanic populations in terms of disease incidence, severe outcomes, and death (Bui et al., 2020; Cates et al., 2020). This burden confers both an urgency to include Hispanic individuals in studies examining neurological, psychological, and cogitative burden over time, as well as challenges in recruiting and retaining a population under great pandemic-associated stress. While the COVID-19 pandemic presented unprecedented challenges for recruitment and retention given the diminished opportunities for face-to-face recruitment and access to in-person community forums, social media and online study recruitment and retention became the main strategies for enrollment in population health research during the pandemic (Ali et al., 2020). While some literature discusses the efficacy of social media recruitment mechanisms, there is limited supporting evidence on the success of these recruitment strategies specific to older adults and racial and ethnic minority communities, particularly during the COVID-19 pandemic (Nash et al., 2017; Remillard et al., 2014; Wasilewski et al., 2019; Whitaker et al., 2017). However, a 2021 study on Alzheimer's prevention research suggested that, compared to non-Hispanic whites, Hispanic individuals prefer engaging with researchers over text versus email (Julbe-Delgado et al., 2021).
The purpose of our study is to describe MindCrowd (MC) Hispanic enrollment across the past decade. In that time period, MC naturally evolved in its design and recruitment approaches making it a unique experiment to examine Hispanic recruitment across a long period of time, including during the COVID-19 pandemic. We use data from MC (https://mindcrowd.org), a large national platform used as the basis for the Precision Aging Network (https://precisionagingnetwork.org/) study to demonstrate recruitment activities across the past decade. In that time period, MC has naturally evolved in its design and recruiting approaches and therefore it can function as a unique experiment to answer questions of how many and trends of Hispanic recruitment into our study. Our aim is to demonstrate that an electronic-based research study (i.e., MindCrowd) would empower enhanced recruitment of Hispanic individuals and recruitment changes during and after the pandemic would also lead to an increase in Hispanic participation.
Methods
MC is a large-scale online national research study that aims to identify potential factors influencing age-related cognitive decline and disease (Huentelman et al., 2020). The MC study was launched in 2013 and has become one of the largest studies of its kind, with over 403,000 participants as of July 2023, from all over the world.
MC1. The time frame for MC1 was January 2013 through March 2020. Website visitors 18 years or older were asked to consent to the study before any data collection via an electronic consent form. The authors confirm they obtained informed consent from each participant and complied with all relevant ethical regulations. The MC research protocol was reviewed and approved by the WCG Institutional Review Board (IRB) with current protocol number 20215906. After consenting to the study and answering five demographic questions (i.e., age, biological sex, years of education, primary language, and country where they reside), participants were asked to complete a web-based svRT task. (Talboom et al., 2021) The option to view the MC website in Spanish or English and to participate in the study and take the exam in Spanish or English has been available since the initiation of MC.
MC-C19. MC-C19 was from April 2020 through May 2022. MC-C19 corresponds to the COVID-19 pandemic emergency time period and included pilot efforts of various activities through social media, email campaigns, online advertisements, and Zoom happy hours in English/Spanish to reach a broad and diverse Hispanic population. Recruitment of Hispanics into health research in the time of a pandemic was a major undertaking for our study. While studies have been conducted to assess different modalities for recruitment, evidence is lacking on the recruitment methods of Hispanics in health research during the COVID-19 pandemic. Our recruitment did not stop during the pandemic. Webinar events and social media recruitment occurred. We showed that this focus had a large effect for Hispanic recruitment and that increasing recruitment of Hispanics is possible.
MC2. MC2 was from July 2022 through July 2023. The main upgrades in features between MC1 and MC2 are a revised webpage and the availability for participation via cell phones. Previous to MC2, only laptop, desktop, and tablet devices were supported. During the data collection period for the MC1 and MC-C19 cohorts, the survey question about ethnicity (whether the user identified as Hispanic) had a default value of False. The user was included in the cohort of Hispanic users if they changed the answer to True, and excluded if the answer value was False. During the data collection period for the MC2 cohort, the survey question about ethnicity (whether the user identified as Hispanic) had a default value of Not Answered (NA). The user could answer True or False. When interpreting the data, NA values were treated as False values in the table for a full count, but as missing in the figures to show as answered figures.
The MC test measures processing speed and verbal memory by conducting simple visual reaction time (svRT) and paired associates learning (PAL) testing. It takes about 10 min to complete and can be taken from any computer or mobile device with an internet connection. The participants completed the MC test by going to the https://mindcrowd.org website. After completing the study consent box, and filling out a demographic survey, the participants complete the svRT and PAL tasks back-to-back. The svRT assessment requires participants to tap the touch screen or press any letter (a-z) on the keyboard as quickly as possible when a pink ball appears. During the PAL task, a 12-item list of word pairs is presented for two seconds each. The participant is instructed to remember as many pairs as possible. During the recall phase, the system displays the first word in the pair, and the participant's task is to type the correct second word. PAL has three total rounds. After completing the svRT and PAL assessments, additional health, medical, and lifestyle questions were asked. Then the system asks if the participant wants to receive their results and more brain studies through email. Finally, people can see their test results compared to previous MC participants who are similar in age, biological sex, and education. It also shows the overall average for all previous MC participants.
Participant Recruitment. In all three periods of MC1, MC-C19, and MC2, we used social media-based algorithmic approaches to specifically target demographics (for example, using Facebook/Meta and Twitter/X marketing approaches). The goal was to increase the reach of our advertising campaigns by dedicating more budget money to such campaigns. In the MC2 period, post-COVID-19 pandemic emergency, we have initiated in-person outreach events at community events, health fairs, consulate offices, and other activities where we can engage with potential participants.
Data Analysis. Weekly MC data on participation and enrollment demographics was assembled for the time period January 1, 2013 through July 31, 2023. For these demographic and enrollment data, means, standard deviations, and percentages were calculated in our descriptive analysis in the three time periods of MC1, MC-19, and MC2. Scatter plots and line graphs were created to show correlations and trends over time. The level of significance was set at 0.05.
Results
From January 2013 to July 2023, a total of 403,633 people participated in MC (Table 1). Of these, 6% identified as Hispanic. The percentage of Hispanics increased from MC1 (4%) to MC-C19 (6%) and MC2 (8%). Overall, the average monthly enrollment in MC2 (8,510/month) was more than three times that of MC1 (2,558/month) and more than two times of MC-C19 (3,893/month). There was a much higher percentage of women in all three study periods of MC, with the percentage of women growing in each study period. Another noticeable difference between MC1, MC-C19, and MC2 is education level. The education attainment levels of MC participants showed an increase in lower levels of education (high school diploma or less) from MC1 (12.5%) to MC-C19 (16.8%) to MC2 (30%).
Characteristics of MindCrowd Participants.
Figure 1 shows the Hispanic recruitment into MC during the three different periods of investigation from 2013 to 2023. MC1, MC-C19, and MC2 are color-coded. Between 2013 and 2016, the regression line is fairly level, but does decrease slightly, with some spikes of enrollment days. In 2017, the regression line starts sloping upwards with a notable spike in enrollment between 2019 and 2020. In MC-C19, the regression line continues upwards, but with a few notable spikes and some decrease between 2021 and 2022. When MC2 started in July 2022, a noticeable disjoint can be seen where recruitment levels start, and the trend line continues on an upwards slope. Beginning in 2017, Figure 1 shows a definite upward trend from MC1 to MC-C19 to MC2 and how MC2 has noticeably higher percentages of Hispanics.

Hispanic recruitment into MindCrowd during the three different periods of investigation. The monthly percentage of the MindCrowd cohort who self-identified as Hispanic (y-axis) is indicated in the context of the date they joined the study (x-axis). The three different time periods of investigation are indicated by their color; orange (through 2020) = MC1, blue (2020- mid 2022) = MC-C19, and pink (mid 2022-2024) = MC2. The regression line that describes how the proportion of Hispanic MindCrowd participants changes over time is depicted in black and the gray shading indicates the 95th percentile confidence intervals.
Figure 2 shows the proportion of Hispanic participants in MC by U.S. state, district, and territory. California, Arizona, Texas and Florida are the four leading states in proportion of Hispanics in MC (over 37%). The increased participation by Hispanic individuals is reflected in the higher proportion of Hispanic residents in these states. In Figure 3, a scatterplot shows the proportion of Hispanic participants in the MC date-based subsets compared to census-reported Hispanic population by U.S. state, district, and territory. Of note for each time period, the correlation coefficient is statistically significant with R = 0.84 (MC1), 0.80 (MC-C19), 0.82 (MC20, and 0.90 (All MC). Notably for MC2, States that had 20% or higher percentage of Hispanics were also recruited at a higher percentage on or above the correlation line.

Proportion of Hispanic participants in MindCrowd by U.S. state, district, and territory

Scatterplot of the proportion of Hispanic participants in the MindCrowd date-based subsets compared to census-reported Hispanic population by U.S. state, district, and territory. Each black dot represents a U.S. state, district, or territory and is localized to coordinates based on the census-reported Hispanic population percentage (y-axis) and the MindCrowd cohort percentage (x-axis). The header of each panel indicates which MindCrowd date-based subset was utilized to generate the plot. The dashed red line indicates the linear correlation line and the R and p values for this correlation are indicated at the top of each panel.
Figure 4 depicts the percentage of Hispanic and Non-Hispanic MC participants as a function of age. These percentages vary across age with the percent of Hispanic participants being higher than the percent of non-Hispanic participants at all younger age groups (until approximately 55 years of age) and lower for Hispanic participants compared to non-Hispanic participants for older age groups (approximately 55 years of age and older). The percentage of Hispanic participation differs between Hispanics and non-Hispanics in a similar trend across all three study periods and all MC. The peaks of increased participation among the Hispanic and non-Hispanic groups also have variations based on ages. Among Hispanic participants, one of the highest participations among age groups are from ages 50 to 55, while among non-Hispanic participants, ages 65 to 70 experience a peak of high participation rates.

Percentage of Hispanic and Non-Hispanic MindCrowd participants as a function of age. The orange lines (top line asymptotic to y-axis) represent the Hispanic participant proportion while the blue lines represent the Non-Hispanic participants. The representative percentage for each sub-group is shown on the y-axis and age is shown on the x-axis. The header of each panel indicates which MindCrowd date-based subset was utilized to generate the plot.
Figure 5 compares Hispanic participation in MC1, MC-C19, and MC2 on a monthly basis. There is an evident increase in overall participation in MC2 and an upward trend overall from MC1, MC-C19, and MC2. Hispanic participation for the second half of the calendar year (June-December) is slightly higher. Recruitment increases steadily through the holiday season with Hispanic participation reaching 20% by December in MC2.

Percentage of Hispanic recruitment into MindCrowd on a monthly basis. The three regression lines illustrate the monthly (x-axis) recruitment of Hispanic participants as a proportion of the total recruited cohort (y-axis) for each indicated date-based group. The three different time periods of investigation are indicated by their color; orange (bottom) = MC1, blue (middle) = MC-C19, and pink (top) = MC2. Shaded areas indicate the 95th percentile confidence intervals.
Discussion
In this paper, we used enrollment data from MC to describe racial and ethnic minority recruitment into a population health research study and showed the need for a diverse study population.(Oh et al., 2015) The hope that precision medicine advances in prevention and treatment reaches all populations requires that racial and ethnic minorities participate in population health and clinical research. In our study, we demonstrated changes in study recruitment during three time periods, MC1, MC-C19, and MC2. The MC1 time period was from when MC was first started (January 2013) through the start of the COVID-19 pandemic (March 2020). MC-C19 covered the COVID-19 pandemic period April 2020 through May 2022, when revisions to the MC webpage and platform were implemented. MC-C19 included pilot efforts of various activities through social media, email campaigns, online advertisements, and Zoom happy hours in English/Spanish to reach a broad and diverse Hispanic population. The main upgrades in features in MC2 are a revised webpage and a change that facilitated participation via cell phone. MC2 occurred from July 2022 through July 2023 for this study.
We sought to describe Hispanic participation in online cognitive research over a 10-year period and to understand whether the participation of Hispanics could be increased through targeted recruitment during MC-C19. Our results showed that online recruitment was effectively able to reach the Hispanic population and the percentage of Hispanic participants increased from MC1, MC-C19, and MC2. Though not at the level of the percentage of Hispanics in the United States (19%), the doubling in growth in percentage of Hispanic participants (4% to 8%) was an encouraging trend and given the technological access issues that have been addressed by MC2. This increasing Hispanic participation throughout the entire MC 10-year study period of 2013–2023 was promising. The trend continues increasing through the post pandemic period. The increase in the slope from 2020 onward are a reflection of the concerted efforts to recruit Hispanic participants into MC. Also, the concordance of U.S. States with a higher percentage of Hispanics having higher Hispanic participation in MC demonstrated that efforts can be improved to increase recruitment in regions with large Hispanic populations. We also saw that younger Hispanics had a higher proportion in MC1 and MC-C19 and that there is a noticeable shift to older Hispanics between the ages of 45–65 in MC2. This shift may be due to the availability of MC on cell phones resulting in an enhanced recruitment of older Hispanics. Additionally, social media (e.g., Spanish MC Facebook website) was targeted to an older population. We note a similar, yet less pronounced, right-ward shift in the curves for the non-Hispanic cohort as well when comparing MC1 with MC-C19 and MC-C19 with MC2.
Among the entire study sample, women participated in a greater percentage than men. This is in line with studies showing greater willingness to participate in research among women and behavioral and preventive clinical trials enrolling a greater proportion of female participants (Otufowora et al., 2021; Steinberg et al., 2021). This is likely due to men and women having different perceptions of risks regarding clinical trials and trust being an essential component of research participation and which may be more difficult for men. (Ding et al., 2007; Striley et al., 2019) Our finding of an increase in the percentage of women and decrease in men in the entire 10-year period is a concern. Efforts are being developed to target men to increase their participation in this research that can have preventive implications. This also speaks to the difficulty of recruiting Hispanic men. In terms of age, a key result was the percentage of participants 50 and older increasing from 48.2% in MC1 to 62.3% in MC-C19 to 81.3% in MC2. This shows the efforts to reach out to older populations were effective during MC-C19 and MC2. The effort to reach populations with education attainment levels of MC participants showed an increase in lower levels of education (high school diploma or less) from MC1 (12.5%) to MC-C19 (16.8%) to MC2 (30%).
In MC2, the key factor was that taking the MC test was available on cell phone reducing potential barriers for participants by opening up our reach to populations that rely more on cell phones than computers or tablets. Additionally, social media outreach efforts increased, including nationally available webinars in 2020 and on ground efforts in 2021 (e.g., mailings in Miami, Mobile Health Unit patient telephone recruitment in Tucson).
Limitations and Strengths. Our study had several limitations. First, in the MC-C19 period, we cannot assess whether increased online/webinar and outreach activities conducted prior to the pandemic would have led to an increased enrollment as well. However, due to the increased online activity during the pandemic year, we assume that there was a larger audience that year that was growing in willingness to receive information via online methods. Another limitation is with the electronic data we have throughout the entire 10-year study period, we are unable to pinpoint the effects of specific recruitment strategies. Often in the past, we have run parallel campaigns and therefore cannot link any particular recruitment to an exact campaign, though we can view recruitment strategies to the month/year shown in the figures and attribute a spike in enrollment to a possible activity, but we cannot tell which recruitment strategy worked best from the spectrum of activities. There were also several strengths to our study. MC has multiple years of recruitment prior to the pandemic and recruitment of older age groups and has had efforts to focus on older age groups and minority populations. Our study allows a historical comparison of MC recruitment numbers prior, during, and after the COVID-19 pandemic and the comparison of the MC1, MC-C19, and MC2 study periods.
Implications. The implications of our research on MC Hispanic enrollment across three distinct periods in the past decade include describing a larger and more diverse study population of Hispanic participants in medical research that will gain knowledge of cognitive aging and dementia. Our aim was to demonstrate that MC would empower enhanced recruitment of Hispanic individuals and recruitment changes during and after the pandemic would also lead to an increase in Hispanic participation. By demonstrating this, it takes us one step closer to our long-term goal for the Hispanic population to benefit from the precision medicine aging initiative and hope for prevention and treatment of cognitive aging diseases in the future.
Future Recommendations. Our future recommendations include developing ways to more specifically evaluate recruitment activities by assigning activity specific tags to MC recruitment events. This process has started, and we are starting to collect data to see how these recruitment numbers for specific activities will look. As part of this effort, we also have begun examining MC2 participation at enhanced recruiting sites versus those that are not, and to compare in-person activities with electronic social media announcements. Additionally, contactability for future follow-up and research is being emphasized in MC2, though that assumes that individuals participating have an email address.
Conclusions
In conclusion, Hispanics joined our web-based research study in large numbers. Findings from MC2 suggest approaches that enable recruitment of Hispanics into population health cognitive research. These approaches include the assurance that a web-based study functions on mobile devices, the study is available in English and Spanish, and bilingual social media communities should be available. Improvement in internet-based Hispanic recruitment over time suggests the potential of innovative strategies to enhance their representation in health research.
Footnotes
Authors’ Note
Precision Aging® Network(PAN) Investigators: A. Aldabergenova, A.Bilgin, A. Bonfitto, A. Box, A. Dolby, A. Feal Rodriguez, A. Glinka, A. Lewis,A. Raju, A. Sidhu, A. Sokan, A. Soldan, A. Teano, A.I Levey, A.J.B. Lee, B.Levin, B. Najafi, B. Rodzon, B. Stark, B.J. LaFleur, C. A. Barnes, C. A. Pettigrew,C. Anderson, C. Camargo, C. -H. Na, C. Rosales, C. Ugonna, C.S. Mitchell, D.Cabral, D. Chambers, D. Coon, D. Kartchner, D. McDougall, D. Metz, D.Sama-Borbon, E. M. Sternberg, F. Taguinod, G. Cay, G. Lieto, H.M. Turner, I.Al-Hussaini, J. Don, J. Pekar, J. R. Runyon, J. Simon, J. Sloan, J. Zhou, J.B.Frye, J.J Lah, K. A. Delgado, K. Chhatlani, K. Ellingson, K. Johnson, K.Khingelova, K. Norton, K. P. Doyle, K. Sanders, K.C. Lark, L. F. Gladulich, L.Ryan, M. Albert, M. Dehghan-Rouzi, M. Flothmann, M. Hay, M. Johnson, M. Lee, M.M. Crespo, M. Mehl, M. Modjeski, M. Naymik, M. Okafor, M.D. De Both, M.J.Huentelman, N. Bhadra, N. Merchant, N. Schork, NK. Chen, P. Bathala, P. F.Worley, P. Pattany, R. Tandon, S. Beres, S. Fox-Rosellini, S. Han, S. Hoscheidt,S. Lohiya, S. Matijevic, S. Merritt, S. Scott, S. Sharma, S. Sockanathan, S.Soto, S.-E. Roh, S.-Y. Kim, T. Ahmed, T. Ariko, T. Nuno, T. Rundek, T. Smith,T. Trouard, T. Yuhas, T.K. Zepeda, V. D. Calhoun, V. M. Dotson, V. Pfeifer, V.Truesdell, X. Li, X. Sun, Y.F. Bolla, Z. Chen, Z. Wang.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This manuscript was funded, in part, by the National Institute on Aging (NIH) (Award# U19AG065169). The content is solely the responsibility of the authors and does not necessarily reflect the official views of the National Institutes of Health.
