Consensus recommendations for usability and acceptability of mobile health autism screening tools

Abstract

Mobile health (mHealth; online phone or tablet-based) screening tools for autism are being increasingly used by parents, health care, and educational providers. However, it is unclear whether mHealth autism screening will improve the inequalities in autism or further help individuals already advantaged in autism care. To try to improve equity in mHealth autism screening, we conducted a modified Delphi consensus panel process with 14 panelists who were medical providers, parents, tool developers, and educational providers. Panelists participated in three asynchronous online voting rounds followed by online discussions. Panelists were provided a preliminary list of 28 recommendations for autism-screening tools that were compiled from previous qualitative interviews in the larger project. In each round, panelists were asked to vote for up to 10 recommendations from the preliminary list and discuss the voting results. After the third round of asynchronous voting, a final round of discussion with all the panelists was held live through Zoom to discuss the final asynchronous voting results. All voting and discussion took place on an online app called Slack. Voting and discussion yielded 19 recommendations in five conceptual categories: transparency, equity, access, product design and user experience, and development process. Tools adhering to these recommendations may increase equitable service use after screening.

Lay abstract

Families may use information online to learn more about autism. Families, health care, and educational providers may use online tools like checklists or web apps to screen for autism (measure whether a young child has autism symptoms or behaviors). However, we do not know whether the online autism-screening tools can be easily used by families and providers. It is possible that families who are culturally diverse, have lower education, or have lower income have trouble using online autism-screening tools. We conducted an online consensus panel with medical/educational providers, parents, and tool developers, asking them to vote and have discussions on suggestions for online screening tools for autism. The voting and discussions resulted in 19 suggestions for changes to make the tools easier to use. The top recommendations included directly linking to sources of information about autism, reading level that is less than or equal to fifth grade, and focusing on the diverse behavior/needs/strengths associated with autism.

Keywords

autism screening autism spectrum disorder Delphi consensus developmental screening disparities mobile health practice guidelines

Introduction

Autism spectrum disorder (ASD) is a lifelong early childhood neurodevelopmental condition. Early identification through autism screening is crucial to increase the likelihood of obtaining intervention support sooner and improve the outcomes for individuals on the autism spectrum. However, despite routine primary care autism screening at well-child visits, delays exist in autism identification: although autism symptoms are usually apparent by age 3 (Ozonoff et al., 2015; Zwaigenbaum et al., 2016), the average diagnosis age in the United States is around 4.5 years (Maenner et al., 2023). Some studies show that families of color, especially those who also have lower income, lower autism-identification rates, and higher severity at the time of diagnosis (Patrick et al., 2025; Shaw et al., 2025). Therefore, current screening is not capturing all children with early signs of autism.

The American Academy of Pediatrics (n.d.) Bright Futures guidelines recommend autism screening at toddler well-child checks. Screening most often involves a parent-completed paper-and-pencil checklist that is filled out and scored in the clinic setting. Over the past 10 years, autism screening has become increasingly common in primary care settings, with a growing number of validated tools available (Felix et al., 2024; Zuckerman et al., 2021). Autism screening in educational settings, such as Early Intervention or Head Start, is becoming more common as well and has been shown to effectively connect children with educational autism services (DeLucia et al., 2024; Sheldrick et al., 2022). However, many health and educational providers remain uncertain about screening tools’ effectiveness and feel inadequately supported in their use (Mazurek et al., 2021). Providers do not always use the tools correctly or refer children to autism-related services after a positive screen (Solgi et al., 2025; Wallis et al., 2020, 2021). In addition, parents are increasingly accessing information about autism independently through community and Internet resources. Thus, autism screening is also commonly taking place outside of usual clinical settings or with usual clinical tools.

Mobile health (mHealth) describes the use of mobile devices for health care use. The use of mobile devices is widespread, even among families of lower socioeconomic status, and mobile devices are more likely to be such families’ only Internet access (Pew Research Center, 2024b). Young women and parents are especially likely to be mobile device users, and many say that would act on recommendations that a mHealth tool advised (Covolo et al., 2021; Van Veen et al., 2019). In addition, parents of young children (age 0–5) use mobile devices at high rates, with women using them more frequently than men (Pew Research Center, 2017). Mothers are also more likely to say they “spend too much time” on their smartphones (Covolo et al., 2021; Pew Research Center, 2020, 2024a). Smartphone and tablet mHealth-based autism screening is a promising area for autism identification given the widespread availability of mobile devices.

mHealth autism-screening tools can be completed by anyone (e.g. a parent or an educational or medical provider) who may have concerns for a child’s development, to identify potential areas of developmental delay(s) and/or early signs of autism. In comparison to clinic-based paper-and-pencil screening checklists, mHealth autism-screening tools may have certain advantages. mHealth screening can be delivered via any mobile device, in children’s natural environments, and with little assistance, giving families access to customized information on the likelihood of autism for their child. The convenience and flexibility to complete the screening in a nonclinical setting could also be helpful for parents to use while juggling other responsibilities. Some tools may allow direct measurement of children’s behaviors (either through video recording, eye tracking, or the child’s interaction with the device), which may allow for earlier and more objective measurement (Tryfona et al., 2016). mHealth-based data collection may be particularly helpful in autism, since many children with autism symptoms have high interest in digital media (Tryfona et al., 2016).

However, while these tools can be helpful for providers and parents, it is not clear whether mHealth autism screeners help everyone equitably. For mHealth screening tools to help all children with signs of autism, they need to be available, usable, and acceptable to populations underserved in autism care, as well as to the health care and educational providers who work with those families (Zuckerman et al., 2023). If mHealth tools are not well suited to the unique needs of low-income families and families of color, widespread use of such tools may exacerbate disparities by providing additional advantages to families who are already advantaged in access to autism care (Nouri et al., 2019; Peek, 2017; Veinot et al., 2018, 2019). For instance, many technology-based interventions are more accessible to young, urban patients seeking care in urban and academic medical centers and do not reach low-income and rural communities (Veinot et al., 2018). Although many low-income families have mobile devices, effectively using mHealth tools on these devices may be challenging due to barriers such as access to Wi-Fi, having limited phone data, the use of older mobile devices, and/or discomfort using technology in general (Real et al., 2018; Veinot et al., 2018). Likewise, barriers that impact mental health intervention use more broadly (e.g. health literacy, stigma around disability and mental health, social support, English proficiency) are also important barriers to the use of mHealth interventions in general and autism interventions specifically (Guerrero & Sobotka, 2022; Kim et al., 2022; Zuckerman et al., 2017).

The purpose of this project is to gain consensus on key features of mHealth autism-screening tools that might benefit families who are historically underserved in autism care (i.e. low-income families, families of color, families with limited English proficiency). Such consensus guidelines could be used by mHealth autism-screening tool developers to improve equitable access to autism care (Maun et al., 2024). They also could be used by health care and educational organizations as a guide for selecting tools for their clinical settings.

This study is part of a comprehensive environmental scan of suitability of mHealth autism-screening tools for families of color, low-income families, and other historically underserved groups within the current system of autism care (grant R21MH120349-01). Previous phases of this project have interviewed mHealth tool developers (i.e. software developers or research team leaders), medical and educational providers, and parents regarding the usability of these tools (Rivas Vazquez et al., 2023; Sanders et al., 2023, 2024; Tangkilisan et al., 2024; Zuckerman et al., 2023).

Since different community members might have different opinions on usability and acceptability of mHealth autism-screening tools, we conducted a consensus panel to try to find common ground among different classes of people who have a role in mHealth autism screening, such as parents, health and educational providers, and autistic adults. The goal of this consensus panel was to isolate a small group of recommendations that was acceptable to many types of people. During the consensus panel, we asked: Which features are most desirable for different groups using the tools? Which features will promote equity?

Methods

We conducted a consensus methodology using a Modified Delphi Panel process (Figure 1) (Hecht, 1979). This process has been previously used by our team to form consensus care recommendations for autism (O’Hagan et al., 2023). The panel included three asynchronous rounds of online discussion and voting using Slack (n.d.) and one live videoconference through Zoom. Panelists were invited via email. The panel took place over the course of 5 weeks, from March 7, 2023 to May 11, 2023. Our research team decided to conduct the panel mostly online to be able to more easily include a diverse group of panelists, by reducing physical and social barriers to participation (e.g. child care for autistic children, social challenges with group discussions for autistic adults). This project was deemed exempt and was approved by the Oregon Health and Science University Institutional Review Board.

Figure 1.

Modified Delphi process used to develop panelists’consensus on recommendations for mHealth tools.

Recruitment

Panelists were recruited using the purposive sampling technique. A total of 25 potential panelists were invited. Nine of these were subjects that had participated in previous phases of this research project, which included health/educational providers, mHealth tool developers, and parents (Rivas Vazquez et al., 2023; Sanders et al., 2024; Tangkilisan et al., 2024; Zuckerman et al., 2023). In addition, 16 potential panelists were sent invitation letters in order to increase the panel size and improve racial/ethnic, gender, and role (e.g. parent, autistic adult) diversity. Invited panelists were sent email invitations explaining the modified Delphi process, voting, compensation information, and their role as panelists. From the 25 invited panelists, 11 declined and/or did not respond. The recruitment yielded a final panel that consisted of two pediatricians, one child psychiatrist, two tool developers, three early intervention providers, and three autistic adults (Table 1). There were also a total of five parents of children diagnosed with autism: Three were solely parents, one was also a tool developer/autism researcher, and one was also an educational provider (Table 1). Some panelists had multiple roles (i.e. parent who was also a tool developer/autism researcher). All interested parties were offered an individual telephone-based orientation to Zoom and Slack, in order to minimize electronic health literacy barriers to participation. Participants used their own mobile devices or computers. Oral language interpretation via Zoom videoconference was offered if needed, although reading and writing in English was a requirement. All panelists were asked to fill out a demographic survey prior to the panel.

Table 1.

Panelist demographics.

	Frequency or mean (n = 13)
Gender
Female	9
Male	4
Non-binary	1
Race
White/Caucasian	9
Asian	2
Black	1
Native American	1
Ethnicity
Caucasian	5
Yoruba	1
Nordic/Irish	1
Chinese	2
Japanese	1
Slavic	1
Latina/Hispanic	2
Age (mean)	42
Personal identifying background
Parent/caregiver for an Autistic individual	5
Healthcare provider	4
Developer of a website or app about Autism	2
Autism researcher	4
Autistic self-advocate	5
Education provider	2
Community information and referral	1
Education background
High school, diploma, or the equivalent	1
Associate degree	1
Bachelor’s degree	3
Master’s degree	5
Professional degree (DO, MD, PA)	2
PhD degree	1
Professional background
Pediatrician/physician	1
Professor	2
EI/ECSE specialist/teacher	2
Business management	1
Patient advocate	1
Scheduler	2
Principal investigator	1
Medical director	1
Program coordinator	1
Graduate student	1
Work in a paid job
Yes	12
No	1
State
Oregon	6
Indiana	1
Missouri	1
Idaho	1
Washington	2
Pennsylvania	1

Respondents could choose more than one option. All questions on the survey were optional, and respondents were not required to answer each question. Therefore, the total responses for each category may not add up to 13 (total respondents). The age range of respondents was 26–59 years of age, the mean was 42 years of age.

Research team

Six individuals in the research team observed, provided additional information, and helped moderate the chat but did not vote on recommendations. The research team members included three general pediatricians, two research assistants, and one undergraduate research intern. The research team also included members of groups historically underserved in autism care, including Black and Latino team members, team members from low-income upbringings, and team members who were themselves or were parents of people with neurodevelopmental conditions.

Orientation

The collaboration app, Slack, was used for the first three rounds of the panel. The research panel private Slack site featured a group chat area, a voting platform that used the app Simple Poll, and private chat areas for parents, autistic adults, autism researchers, health care providers, and the research team. If panelists played multiple roles (for instance, a health care provider who was also a parent of an autistic child), then they were invited to multiple chat areas. To encourage non-academic partners to participate, 1 parent and 1 autistic adult were designated “discussion leaders” who facilitated the private chat rooms and encouraged non-academic partners to comment and vote. These discussion leaders received additional compensation.

One week prior to the first round, panelists were invited to the private Slack site. An email was sent to the panelists detailing how to sign in on the site; we also provided technical support via email to ensure all panelists successfully joined the site. To further orient the panelists, we asked them to introduce themselves and practice voting on a survey topic unrelated to the topic of mHealth tools within the site.

We then posted lay-language, multimedia summaries of research on screening tool usability/accessibility from prior phases of the project, and a list of 28 preliminary design recommendations from our prior research (Supplemental Table 1). Multimedia summaries included small clips from previous recorded in-person interviews and one-page summaries of prior research. Panelists were invited to review the materials and comment on them.

First round

Panelists were given 11 days to review the initial 28 statements and engage in conversation. All 28 statements were entered into a poll on Slack as options to vote on with the prompt “Please choose from the following suggestions you think are most essential to improving online autism-screening tools. Please read all choices first, then vote for UP TO 10 choices.” The statements were inserted in no particular order.

Panelists were asked to make at least 3 comments on Slack per round. Comments could be anything but were often suggestions on combining or altering the wording of the statements, or suggestions for additional recommendations not listed. Statements that received 0-3 votes were not included in the next round of voting for the first two rounds and statements receiving 0 votes in third round were excluded from the final recommendations discussed in the Zoom video conference.

A total of 25 statements were curated at the end of the first round. Twenty-four of the original statements were kept, multiple statements were reworded, and one new additional statement from the panelists’ discussion was added. Decisions about integrating suggestions from panelists were discussed among the research team until consensus was reached.

Second round

A new poll was created with the 25 statements derived from the previous round and were presented to vote on for the second round. The same poll prompt was used as the previous round. The panelists had 10 days to vote and make any suggestions on changes or additions to the statements. Based on a participant panel suggestion, a grouping was made of five different categories the panelists stated were most relevant to the development of the tools. Each recommendation was assigned one grouping per panelist recommendations, for a total of 20 statements to vote on in the next round. Ten of the original 28 statements were included, and the remaining were created by panelists via combining and rewording statements.

Third round

There were 20 statements in the third round. Five separate polls, with the themes “Transparency,” “Equity,” “Access,” “Product Design and User Experience,” and “Development Process,” were created with groupings suggested by panelists in round two. Each poll included statements based on the formatted suggestion from the panelists. The result of this yielded 19 recommendations with at least one vote, leaving one statement that received zero votes to be excluded from the final list of recommendations. Nine statements were included as they were originally written in the preliminary round (Supplemental Table 1). The remaining 10 statements were either reworded or modified collaboratively based on panelist comments.

Final round: videoconference to finalize recommendations

The panelists participated in a one-and-a-half-hour videoconference over Zoom. For a portion of the meeting, research team organized four breakout rooms for the different stakeholders: parents, developers, autistic adults, and providers. In each breakout room, the panelists reviewed the final list of recommendations from round three voting. They were asked to reflect on the final recommendations for the wording and the definition of the statements, and they had the opportunity to suggest any changes. Final definitions of the statements were created by the study team and were reviewed by a parent, an autistic adult, and a developer who is also a provider from the panel, prior to publication.

Results

Panelists

Panelist characteristics are shown in Table 1. Of the 14 panelists, 14% were parents, 50% were health or educational providers, 21% were autistic adults, and 14% were developers. Several panelists played multiple roles. Sixty-three percent were female, and 68% were White race. One panelist spoke Mandarin/Chinese and could only read and write in English but could not speak it. They were able to participate in voting and commenting processes on Slack unassisted, but they were provided with an interpreter for support with oral communication in the final round. The remainder of the panelists spoke English. All panelists were compensated for each round in which they made at least three comments during the asynchronous discussions.

All panelists participated in the voting polls all three rounds. For each round 14, 14, and 13 panelists, respectively, made at least 3 comments. There were a total of 50, 70, and 45 comments/suggestions during each round. respectively.

Voting results

The top 19 recommendations that were considered most important to panelists along with what each recommendation means are presented in Table 2. The top three highest ranked statements were “tools should directly link to sources of information about autism and/or community resources, and should allow parents to directly refer the child to child development programs (e.g. Early Intervention),” “tool’s reading level should be fifth grade or less and should be easily understandable for parents with elementary level math education,” and “tools should be developed for the wide range and variability of behaviors, needs, strengths, and individual experiences associated with autism, including both children with high support needs and children with subtle, atypical, or inconsistent behaviors and presentation” (Table 3).

Table 2.

Final recommendations for mobile online autism spectrum disorder screening tools from the consensus panel.

Group 1: Transparency
1.	Tool should not include links to advertisements for commercial products or services
2.	Tool should clearly explain its objective(s) on the first screen
3.	Tool should request the minimal level of personal data needed to function as a screener and be transparent about how it uses child/parent data
Group 2: Equity
4.	Tool should explain what autism is, include positive characteristics of children with autism, and incorporate the perspectives of autistic people in information provided to families
5.	Tool should directly link to sources of information about autism and/or community resources and allow parents to directly refer the child to local child-development programs (e.g. Early Intervention)
6.	Tool should have gender, racial/ethnic, and cultural diversity in videos, questions, and graphics; should show humility with respect to minority individuals and cultures
7.	Tool reading level should be fifth grade or less; should be easily understandable for parents with elementary-level math education
8.	Fees (costs) to the family should be apparent up-front
Group 3: Access
9.	Tool should specify a child age range that it targets
10.	Tool should be accessible on multiple mobile platforms (e.g. iOS, Android) and should be version-compatible with older and less-expensive devices
11.	Parents should be able to complete the tool on average in <10 min; trained personnel should be able to interpret the tool on average in <5 min
Group 4: Product design and user experience
12.	Tool should avoid stigmatizing words (e.g. “normal,” “abnormal,” “risk”)
13.	Tool should provide user support resources (e.g. help when you get stuck or are confused)
14.	Tool should not associate red color, bolding, or other stressful signals with a positive test result
Group 5: Development process
15.	Tool’s test properties (i.e. accuracy) should be studied in the under-resourced communities and common languages in the countries in which it is marketed
16.	Tools should be developed for the wide range and variability of behaviors, needs, strengths, and individual experiences associated with autism, including both children with high support needs and children with subtle, atypical, or inconsistent behaviors and presentation
17.	Tools should be developed, tested, and made available in the most common languages in the countries in which they are marketed
18.	Funders should explicitly request that new mHealth autism screeners include equity considerations and provide funding for this purpose
19.	Tool design should be participatory and include both parents and autistic individuals

Table 3.

Top 10 most popular recommendations from final voting.

Recommendation	Number of votes (n = 14)
Tool should directly link to sources of information about autism and/or community resources and allow parents to directly refer the child to child-development programs (e.g. Early Intervention)	12
Tool reading level should be fifth grade or less; should be easily understandable for parents with elementary-level math education	12
Tools should be developed for the wide range and variability of behaviors, needs, strengths, and individual experiences associated with autism, including both children with high support needs and children with subtle, atypical, or inconsistent behavior and presentation	11
Tool design should be participatory and include both parents and autistic individuals	10
Tool’s test properties (i.e. accuracy) should be studied in the under-resourced communities and common languages in the countries in which it is marketed	10
Tool should avoid stigmatizing words (e.g. “normal”, “abnormal”, “risk”)	9
Tool should be accessible on multiple mobile platforms (e.g. iOS, Android)	9
Tool should have gender, racial/ethnic, and cultural diversity in videos, questions, and graphics; should show humility with respect to minority individuals and cultures (e.g. explaining unfamiliar concepts such as “joint attention”, portraying families of color, with visible disability and diverse family structures)	9
Tool should clearly explain its objective on the first screen	9
Tool should specify a child age range that it targets	8

The final recommendations were categorized into five groups by the panel: transparency, equity, access, product design and user experience, and development process. The panel collaboratively named the groups as follows and decided on which statements fit into which grouping during the second round of voting and defined the group titles during the in-person discussion at the end.

Transparency means that the tool’s purpose should be clear to its users. For example, panelists agreed that a tool “should clearly explain its objective(s) on the first screen” and “should not include links to advertisements for commercial products or services.”

Equity means that the tool is created with consideration of the varying needs of diverse populations, especially populations that have been marginalized in autism care. Examples of this being “tool reading level should be fifth grade or less; should be easily understandable for parents with elementary level math education” (promoting equity for families with lower educational attainment) and “fees (costs) to the family should be apparent up-front” (promoting equity for lower-income families).

Access means that it should be easy for intended users to successfully use the tool, including eligibility, time burden, and time for interpretation of results. Examples of this category are “tool should specify a child age range that it targets” and “tool should be accessible on multiple mobile platforms (e.g. iOS, Android) and should be version-compatible with older and less expensive devices.”

Product design and user experience means that tool features and content should avoid using stigmatizing terminology or design features. Two examples from this category are “tool should avoid stigmatizing words (e.g. “normal,” “abnormal,” “risk”)” and “tool should not associate red color, bolding, or other stressful signals with a positive test result.”

Development process means using a participatory tool-development process that includes thoughts and opinions of parents, medical providers, and community members. Examples of what panelists agreed this could look like in a tool are “tools should be developed, tested, and made available in the most common languages in the countries in which they are marketed” and “tool design should be participatory and include both parents and autistic individuals.”

Discussion

This study used a modified Delphi method to develop consensus for mHealth autism-screening tool design features, to meet the needs of multiple types of people who would encounter and utilize such tools. This study is the first to integrate multiple points of view on an emerging technology that is increasingly used to identify autism early in a child’s development, and how this technology is viewed by different affected social groups.

Overall, panelists prioritized recommendations that supported family familiarity with autism and systems of autism care. In particular, panelists expressed that tools need to link to resources with information about autism and community resources, with emphasis on local interventions and reputable resources. Panelists discussed that providing these reputable sources within the context of a roadmap, rather than blankly supplying a list of resources, would provide parents with a clearer understanding for what is down the line after completing the tool. This could be an important first step for developers to try integrating linked sources for parents and providers to locate their state’s early-intervention services. Although barriers to download medical apps have been previously cited as a reason for not using mHealth technology, the panel’s consensus also highlights the known “technostress” phenomenon (stress due to constant use of technology and difficulty adapting to it) experienced by patients and caregivers when relating to technology (Real et al., 2018; Tryfona et al., 2016). User-friendly design is an important future direction in reducing this stress.

The panelists also valued screening tool features that created clarity for users. Within the 10 most popular recommendations, there was clear consensus that prioritizing the need for the tool to be more accessible and understandable to parents and families; for instance, top choices in the last round were that the tool should “clearly explain its objective on the first screen, and have reading level fifth grade or less and easily understandable for with elementary level math education,” “avoid stigmatizing words,” and “tool should specify a child age range that it targets.” Creating mHealth tools that are easily adopted by all individuals may increase confidence for individuals with less technology experience (Veinot et al., 2018). One such approach might include using plain language guidelines, which have been promoted by the U.S. Federal Government and other entities as a means of improving health care access. Plain language guidelines include the importance of knowing the “expertise and interest of your average reader” versus for the “experts” on the topic, using concrete versus abstract words, and conducting usability testing (Plainlanguage.Gov, n.d.). In addition, the guidelines note that web content is clear when users can find, understand, and use it to meet their needs (Plainlanguage.Gov, n.d.). These guidelines, along with the panel recommendations for Autism mHealth tools, can help clarify actionable items for mHealth tool developers.

These recommendations have importance for mHealth autism-screening tool designers. Although the recommendations are aspirational and have varying implementation challenges, many of the recommendations could be implemented at relatively low cost. For instance, many product design and user experience features, such as avoiding the word “abnormal” or not associating the color red with a positive test result, could be implemented with very low cost. Other features (e.g. explaining tool objectives on the first screen) are general features of good product design which would benefit all users.

Although designed for mHealth tools, many of these recommendations would be useful for paper-and-pencil/traditional autism-screening tools as well (e.g. reading level, stigmatizing language). Incorporating these recommendations would enhance screening fidelity and performance in underserved populations in many settings. Likewise, some of the process recommendations (e.g. involving autistic individuals and parents in the screening tool design process) would lead to development of tools that have greater broad acceptability to families. Refining existing screening tools may also help increase provider confidence with implementing novel screening and follow-up practices. Developing workflow around new screening tools will also be important for community settings as they present new challenges such as integrating information across device platforms and maintenance of tech support for screening. Enhancing provider confidence through more accessible and reliable tools may contribute to reduced workflow barriers and more consistent referral practices following positive screens.

These consensus recommendations could have a variety of policy uses. For instance, health care institutions or Medicaid Accountable Care Organizations could cover payment for mHealth autism screening based on degree of adherence to these consensus recommendations, as a way to promote service use equity. Individual providers or health systems could use these guidelines to compare or vet different mHealth autism-screening tools when deciding which ones to use. Finally, community advocacy organizations could use these standards to evaluate tools that they are considering endorsing, hosting on their websites, or linking to an electronic media.

This study had limitations. The consensus panel was conducted entirely online in order to make it accessible for panelists who were spread over a wide geographic area and had other competing responsibilities. In addition, we felt that the mostly asynchronous online format might “level the playing field” for autistic individuals who can struggle with in-person interactions. It also fit with the overall project goal of evaluating tools that were themselves online. We did offer digital orientations to anyone who wanted to participate; however, this format may have excluded those who could not get online or who did not feel comfortable using Slack. During our online Zoom discussion, not all panelists may have felt equally empowered to speak, which we tried to mitigate by having separate breakout groups for parents, providers, autistic adults, and tool developers. Although we invited individuals from diverse ethnic and racial backgrounds to join the panel and share their perspectives, the majority of our panelists were White. Therefore, the results may not fully represent the views or preferences of individuals from varied racial or ethnic backgrounds. One panelist was a Chinese/Mandarin speaker who was being supported with a simultaneous interpreter; however, technological issues made this challenging during the final in-person meeting. Similarly, we tried to accommodate different disabilities by allowing for multiple formats such as closed captions during the Zoom discussion, but engagement still may have been hard for some people. There may have also been positivity bias because the panelists knew the discussion was being monitored by the research team. The panelists had to be able at least write in English which could have been challenging for those with lower intellectual ability or English proficiency.

In summary, this consensus panel process prioritized mHealth tool features that would decrease barriers to tool use and increase user confidence in mHealth screening tools, emphasizing the need for tools to support families more immediately such as directly linking sources of information about autism to designing the tool purposely in navigating systems of autism care. The most desired equity-promoting features were those that would increase access to all individuals, including those who have a lower familiarity with autism and/or mobile technology. If a long-term goal is to improve equity in access to autism services, improving families’ first contact with these services—screening—is a key step. In the future, we hope to use these recommendations to assess whether tools that more closely adhere to these recommendations lead to more equitable service use after screening. It is our hope that this investigation will lead to more careful consideration of equity impact at all stages of mHealth autism-screening tool development and use.

Supplemental Material

sj-docx-1-aut-10.1177_13623613251360276 – Supplemental material for Consensus recommendations for usability and acceptability of mobile health autism screening tools

Supplemental material, sj-docx-1-aut-10.1177_13623613251360276 for Consensus recommendations for usability and acceptability of mobile health autism screening tools by Gabriella Tangkilisan, Luis Rivas Vazquez, Plyce Fuchu, Benjamin Sanders, Jill K Dolata, Steven Bedrick, Eric Fombonne, Sarabeth Broder-Fingert and Katharine E Zuckerman in Autism

Footnotes

Acknowledgements

The research team would like to acknowledge panelists Carl Peterson and Ekatrina Nadeskin for their contributions to data collection and this article.

ORCID iDs

Gabriella Tangkilisan

Benjamin Sanders

Eric Fombonne

Katharine E Zuckerman

Author contributions

Gabriella Tangkilisan: Data curation; Formal analysis; Methodology; Writing—original draft.

Luis Rivas Vazquez: Data curation; Formal analysis; Funding acquisition; Methodology; Project administration; Writing—review & editing.

Plyce Fuchu: Funding acquisition; Methodology; Writing—review & editing.

Benjamin Sanders: Formal analysis; Methodology; Writing—review & editing.

Jill K. Dolata: Methodology; Writing—review & editing.

Steven Bedrick: Methodology; Writing—review & editing.

Eric Fombonne: Methodology; Writing—review & editing.

Sarabeth Broder-Fingert: Conceptualization; Funding acquisition; Methodology; Project administration; Supervision; Writing—review & editing.

Katharine E. Zuckerman: Conceptualization; Data curation; Formal analysis; Funding acquisition; Methodology; Supervision; Validation; Writing—original draft.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded by the National Institute of Mental Health grant R21MH11299. BS involvement was supported by a National Library of Medicine training grant 5T15LM007088. GT involvement was supported by BUILD-EXITO (National Institutes of Health Common Fund and Office of Scientific Workforce Diversity grants UL1GM118964, RL5GM118963, and TL4GM118965). JKD involvement was supported by the Northwest Native American Center of Excellence (Health Research and Services Administration grant D34HP31026).

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: SB-F is on the Scientific Advisory Board of EarliTec Diagnostics Inc. There are no other conflicts of interest to disclose.

Supplemental material

Supplemental material for this article is available online.

References

American Academy of Pediatrics. (n.d.). Bright futures. https://www.aap.org/en/practice-management/bright-futures/

Covolo

Zaniboni

Roncali

Mapelli

Ceretti

Gelatti

(2021). Parents and mobile devices, from theory to practice: Comparison between perception and attitudes to 0–5 year old children’s use. International Journal of Environmental Research and Public Health, 18(7), 3440. https://doi.org/10.3390/ijerph18073440

DeLucia

E. A.

Dike

Andrzejewski

T. M.

McDonnell

C. G.

(2024). Autism screening practices in preschools and early childcare centers: A systematic review. Review Journal of Autism and Developmental Disorders, 11(1), 1–20. https://doi.org/10.1007/s40489-022-00323-9

Felix

Deavenport-Saman

Stavros

Farboodi

Durazo-Arvizu

Garcia

Yin

Gera

M. P.

(2024). Standardizing and improving primary care-based electronic developmental screening for young children in federally qualified health center clinics. Maternal and Child Health Journal, 28(10), 1716–1725. https://doi.org/10.1007/s10995-024-03970-y

Guerrero

M. G. B.

Sobotka

S. A.

(2022). Understanding the barriers to receiving autism diagnoses for Hispanic and Latinx families. Pediatric Annals, 51(4), e167–e171. https://doi.org/10.3928/19382359-20220322-03

Hecht

A. R.

(1979). A modified Delphi technique for obtaining consensus on institutional research priorities. Community Junior College Research Quarterly, 3(3), 205–214. https://doi.org/10.1080/0361697790030301

Kim

Dababnah

Wang

Reyes

(2022). Diagnostic process and barriers among Chinese-American and Korean-American parents of children with autism. Journal of Developmental & Behavioral Pediatrics, 43(6), 327–334. https://doi.org/10.1097/DBP.0000000000001070

Maenner

M. J.

Warren

Williams

A. R.

Amoakohene

Bakian

A. V.

Bilder

D. A.

Durkin

M. S.

Fitzgerald

R. T.

Furnier

S. M.

Hughes

M. M.

Ladd-Acosta

C. M.

McArthur

Pas

E. T.

Salinas

Vehorn

Williams

Esler

Grzybowski

Hall-Lande

Shaw

K. A.

(2023). Prevalence and characteristics of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2020. MMWR Surveillance Summaries, 72(2), 1–14. https://doi.org/10.15585/mmwr.ss7202a1

Maun

Fabri

Trevorrow

(2024). Participatory methods to engage autistic people in the design of digital technology: A systematic literature review. Journal of Autism and Developmental Disorders, 54(8), 2960–2971. https://doi.org/10.1007/s10803-023-06015-5

10.

Mazurek

M. O.

Kuhlthau

Parker

R. A.

Chan

Sohl

(2021). Autism and general developmental screening practices among primary care providers. Journal of Developmental & Behavioral Pediatrics, 42(5), 355–362. https://doi.org/10.1097/DBP.0000000000000909

11.

Nouri

S. S.

Avila-Garcia

Cemballi

A. G.

Sarkar

Aguilera

Lyles

C. R.

(2019). Assessing mobile phone digital literacy and engagement in user-centered design in a diverse, safety-net population: Mixed methods study. JMIR mHealth and uHealth, 7(8), Article e14250. https://doi.org/10.2196/14250

12.

O’Hagan

Krauss

S. B.

Friedman

A. J.

Bartolotti

Abubakare

Broder-Fingert

Augustyn

(2023). Identifying components of autism friendly health care: An exploratory study using a modified Delphi method. Journal of Developmental & Behavioral Pediatrics, 44(1), e12–e18. https://doi.org/10.1097/DBP.0000000000001139

13.

Ozonoff

Young

G. S.

Landa

R. J.

Brian

Bryson

Charman

Chawarska

Macari

S. L.

Messinger

Stone

W. L.

Zwaigenbaum

Iosif

(2015). Diagnostic stability in young children at risk for autism spectrum disorder: A baby siblings research consortium study. Journal of Child Psychology and Psychiatry, 56(9), 988–998. https://doi.org/10.1111/jcpp.12421

14.

Patrick

M. E.

Williams

A. R.

Shaw

K. A.

Durkin

Hall-Lande

Vehorn

Hughes

(2025). Social vulnerability and the prevalence of autism spectrum disorder among 8-year-old children, Autism and Developmental Disabilities Monitoring Network, 2020. Annals of Epidemiology, 104, 8–14. https://doi.org/10.1016/j.annepidem.2025.02.014

15.

Peek

M. E.

(2017). Can mHealth interventions reduce health disparities among vulnerable populations? Diversity & Equality in Health and Care, 14(2), 100091. https://doi.org/10.21767/2049-5471.100091

16.

Pew Research Center. (2017). Internet, broadband fact sheet. http://www.pewinternet.org/fact-sheet/internet-broadband/

17.

Pew Research Center. (2020). Parenting children in the age of screens. https://www.pewresearch.org/internet/2020/07/28/parents-attitudes-and-experiences-related-to-digital-technology/

18.

Pew Research Center. (2024a). Internet, broad fact sheet. https://www.pewresearch.org/internet/fact-sheet/internet-broadband/?tabItem=d5edf003-5858-4269-89c5-f2889ecf7951

19.

Pew Research Center. (2024b). Mobile fact sheet [Dataset]. https://www.pewresearch.org/internet/fact-sheet/mobile/

20.

Plainlanguage.gov. (n.d.). https://www.plainlanguage.gov/guidelines/

21.

Real

F. J.

DeBlasio

Rounce

Henize

A. W.

Beck

A. F.

Klein

M. D.

(2018). Opportunities for and barriers to using smartphones for health education among families at an urban primary care clinic. Clinical Pediatrics, 57(11), 1281–1285. https://doi.org/10.1177/0009922818772157

22.

Rivas Vazquez

Tangkilisan

Sanders

B. W.

Bedrick

Zuckerman

K. E

. (2023, May 1). Design and usability of mHealth autism screening tools for families underserved in autism care: Parent perspectives. Pediatric Academic Societies. https://2023.pas-meeting.org/fsPopup.asp?PosterID=564789&mode=posterinfo

23.

Sanders

B. W.

Bedrick

Broder-Fingert

Brown

S. A.

Dolata

J. K.

Fombonne

Reeder

J. A.

Rivas Vazquez

L. A.

Fuchu

Morales

Zuckerman

K. E.

(2023). Mobile and online consumer tools to screen for autism do not promote equity. Autism, 27(3), 714–722. https://doi.org/10.1177/13623613221114280

24.

Sanders

B. W.

Bedrick

Rivas Vazquez

Broder-Fingert

Brown

Dolata

J. K.

Fombonne

Fuchu

Zuckerman

K. E.

(2024, May 3). Mobile apps and websites that screen for autism are of low quality. Pediatrics Academic Societies. https://2024.pas-meeting.org/fsPopup.asp?PosterID=646464&mode=posterInfo

25.

Shaw

K. A.

Williams

Patrick

M. E.

Valencia-Prado

Durkin

M. S.

Howerton

E. M.

Ladd-Acosta

C. M.

Pas

E. T.

Bakian

A. V.

Bartholomew

Nieves-Muñoz

Sidwell

Alford

Bilder

D. A.

DiRienzo

Fitzgerald

R. T.

Furnier

S. M.

Hudson

A. E.

Pokoski

O. M.

. . .Maenner

M. J.

(2025). Prevalence and early identification of autism spectrum disorder among children aged 4 and 8 years—Autism and Developmental Disabilities Monitoring Network, 16 sites, United States, 2022. MMWR Surveillance Summaries, 74, 1–22.

26.

Sheldrick

R. C.

Carter

A. S.

Eisenhower

Mackie

T. I.

Cole

M. B.

Hoch

Brunt

Pedraza

F. M.

(2022). Effectiveness of screening in early intervention settings to improve diagnosis of autism and reduce health disparities. JAMA Pediatrics, 176(3), 262. https://doi.org/10.1001/jamapediatrics.2021.5380

27.

Slack. (n.d.). Computer software. https://slack.com/

28.

Solgi

Calub

Feryn

Hoang

Fombonne

Matushak

Bush

Zuckerman

K. E.

(2025). Predictors of early intervention referral after a positive developmental screen in community primary care clinics. Academic Pediatrics, 25(3), 102591. https://doi.org/10.1016/j.acap.2024.10.003

29.

Tangkilisan

Rivas Vazquez

Fuchu

Sanders

B. W.

Broder-Fingert

Zuckerman

K. E.

(2024, May 3). Consensus recommendations for usability and acceptability of mobile health autism screening tools. Pediatric Academic Societies. https://2024.pas-meeting.org/searchGlobal.asp?mode=Posters&SearchQuery=tangkilisan

30.

Tryfona

Oatley

Calderon

Thorne

(2016). M-health solutions to support the national health service in the diagnosis and monitoring of autism spectrum disorders in young children. In Antona

Stephanidis

(Eds.), Universal access in human-computer interaction: Users and context diversity (Vol. 9739; pp. 249–256). Springer. https://doi.org/10.1007/978-3-319-40238-3_24

31.

Van Veen

Binz

Muminovic

Chaudhry

Rose

Calo

Rammal

J.-A.

France

Miller

. (2019). Potential of mobile health technology to reduce health disparities in underserved communities. Western Journal of Emergency Medicine, 20(5), 799–802. https://doi.org/10.5811/westjem.2019.6.41911

32.

Veinot

T. C.

Ancker

J. S.

Cole-Lewis

Mynatt

E. D.

Parker

A. G.

Siek

K. A.

Mamykina

(2019). Leveling up: On the potential of upstream health informatics interventions to enhance health equity. Medical Care, 57(Suppl. 2), S108–S114. https://doi.org/10.1097/MLR.0000000000001032

33.

Veinot

T. C.

Mitchell

Ancker

J. S.

(2018). Good intentions are not enough: How informatics interventions can worsen inequality. Journal of the American Medical Informatics Association, 25(8), 1080–1088. https://doi.org/10.1093/jamia/ocy052

34.

Wallis

K. E.

Davis Rivera

L. B.

Guthrie

Bennett

A. E.

Mandell

D. S.

Miller

J. S.

(2021). Provider responses to positive developmental screening: Disparities in referral practices? Journal of Developmental & Behavioral Pediatrics, 42(1), 23–31. https://doi.org/10.1097/DBP.0000000000000855

35.

Wallis

K. E.

Guthrie

Bennett

A. E.

Gerdes

Levy

S. E.

Mandell

D. S.

Miller

J. S.

(2020). Adherence to screening and referral guidelines for autism spectrum disorder in toddlers in pediatric primary care. PLOS ONE, 15(5), Article e0232335. https://doi.org/10.1371/journal.pone.0232335

36.

Zuckerman

K. E.

Chavez

A. E.

Wilson

Unger

Reuland

Ramsey

King

Scholz

Fombonne

(2021). Improving autism and developmental screening and referral in US primary care practices serving Latinos. Autism, 25(1), 288–299. https://doi.org/10.1177/1362361320957461

37.

Zuckerman

K. E.

Lindly

O. J.

Reyes

N. M.

Chavez

A. E.

Macias

Smith

K. N.

Reynolds

(2017). Disparities in diagnosis and treatment of autism in Latino and non-Latino white families. Pediatrics, 139(5), Article e20163010. https://doi.org/10.1542/peds.2016-3010

38.

Zuckerman

K. E.

Rivas Vazquez

L. A.

Morales Santos

Fuchu

Broder-Fingert

Dolata

J. K.

Bedrick

Fernandez

Fombonne

Sanders

B. W.

(2023). Provider perspectives on equity in use of mobile health autism screening tools. Autism, 28, 1947–1958. https://doi.org/10.1177/13623613231215399

39.

Zwaigenbaum

Bryson

S. E.

Brian

Smith

I. M.

Roberts

Szatmari

Roncadin

Garon

Vaillancourt

(2016). Stability of diagnostic assessment for autism spectrum disorder between 18 and 36 months in a high-risk cohort. Autism Research, 9(7), 790–800. https://doi.org/10.1002/aur.1585

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB