ADHDCoach—a virtual clinic for parents of children with ADHD: Development and usability study

Abstract

Attention-deficit/ hyperactivity disorder (ADHD) is one of the most common mental health problems in childhood. Despite the fact that evidence-based treatments exist, behavioral parent training programs are the gold standard in the care of children with ADHD, a significant percentage of parents of children with ADHD do not access such interventions. Internet-delivered interventions are effective for a range of mental health problems, however, there is limited research conducted on the efficacy of such interventions in the treatment of ADHD.

Objective: The aim of this study is to present the development and feasibility of an Internet-delivered intervention for parents of children with ADHD. Methods: The intervention was based on Behavioral Parent Training and Rational Emotive Behavior Therapy. Participants were mental health specialists (N = 16) and parents of children diagnosed with ADHD (N = 24). Results: Our results indicated high usability and parental satisfaction with the intervention. Conclusion: In conclusion, an Internet-delivered intervention addressed to parents of children diagnosed with ADHD is a promising approach. Future research should investigate the efficacy of this Internet-delivered intervention in a randomized controlled trial.

Keywords

online technology digital health mHealth Psychology Internet

Introduction

Attention-deficit/ hyperactivity disorder (ADHD) is a common mental health issue in children with a significant impact on children, families and society. A considerable majority of children do not have access to mental health services despite the existence of evidence-based treatment, with pharmacologic and non-pharmacologic options available.¹ Behavioral parent training (BPT) is recommended as first-line treatment for children with ADHD,² however, parents encounter multiple obstacles in gaining access³ to such programs and dropout rates are high⁴ even among those who do gain access to BPT.

Technology has the potential to increase access to evidence-based treatments and surpass existing gaps. For example, Internet-delivered interventions are associated with moderate to large effect sizes for anxiety and depressive symptoms in adults.⁵ Online parenting programs are effective in reducing child behavioral problems.⁶ As regards mobile applications, despite the fact that there is a proliferation of such apps in commercially available stores (e.g. Apple store, Google Play), research regarding ADHD apps' efficacy/ effectiveness lags behind.⁷ New developments in technology (e.g. Internet-delivered interventions, mobile applications) have been used in the treatment of mental health problems for a long period of time, however, only recently such approaches have been considered in the treatment of ADHD.

A recent meta-analysis⁸ indicated that online treatments for ADHD can be a viable option with a moderate effect size on attention deficits (SMD = −.73) and social function (SMD = −.59) when compared to waitlist controls. These results, however, need to be interpreted carefully given that only six studies were included, three conducted with adult populations and three with children. There is preliminary research investigating the efficacy of online behavior parenting training^9–11 or some programs are still under investigation,¹² however, more research needs to be conducted in order to identify predictors of acceptability given that certain components of parent training could be more important than others¹³ in terms of parental outcomes.

In a pilot randomized trial,¹⁰ DuPaul and collaborators investigated the efficacy of an online parent training program as compared to face-to-face parent training and a waitlist control for parents of preschool children at risk of ADHD (n = 47 families). The intervention consisted of 10 sessions of BPT. The online program consisted of an initial face-to-face group meeting in which parents were presented the program (participants from both groups participated in this first session) and the remaining nine sessions were delivered online weekly. Parents allocated in the online group received technical assistance to access the program. In addition, parents in the online condition got weekly telephone calls from a research assistant to review program implementation and answer questions regarding intervention methods. Both active conditions resulted in improved treatment knowledge, and child behavior; session attendance (an average of 80%) was higher than in previous studies. Short-term effects of online parent training on child ADHD symptoms were also found in a randomized controlled trial where parents (n = 53) of preschool children with elevated ADHD symptoms were allocated either to the online self-help experimental group or control (delayed treatment).¹¹ The online self-help group received access to a positive parenting program structured into eight modules, delivered over 16 weeks. Two telephone consultations with a program facilitator were also included in order to tailor the strategies to specific family situations and improve attendance. Results indicated significant reductions in ADHD symptoms, improvements in parent-related outcomes (parental stress, depression, self-efficacy) as rated by mothers and improvements in prosocial behavior according to teacher ratings. Results were maintained at 6-months follow-up, however, only for mother-related variables (parental efficacy, depression, stress) and not for child ADHD, which remained at the same level as at posttest for the experimental group. It is possible that the lack of between groups difference in ADHD symptoms at follow-up could be explained by the self-help nature of the online program, or the small sample size which could limit the detection of difference between the two groups.

Different results emerged in the literature when the included participants were not parents of children at risk of ADHD but parents of children diagnosed with ADHD, who were referred for treatment. In a pilot non-inferiority trial, Breider and collaborators⁹ compared the efficacy of a blended intervention (n = 11) with a face-to-face (n = 11) BPT in a sample of parents of school-age children diagnosed with ADHD, referred for treatment. The blended intervention consisted of six modules delivered online and face-to-face contacts in the clinic both at the beginning and at the end of treatment, as well as during the treatment (evaluation contacts regarding the progress during the program). The intervention was structured in four phases and was delivered over 20 weeks. Results indicated that the blended condition was not non-inferior, and the dropout rate was higher (90.9% in the blended condition vs. 40% in the face-to-face group before training phase 4, designed for evaluation and generalization). In fact, most of the participants from the blended condition (72.7%) dropped the intervention in phase 1 (Introduction and psychoeducation). Moreover, parental efforts, as rated by clinicians, were considered lower in the blended condition as compared to the face-to-face condition. These results need to be interpreted cautiously given the small sample size. Also, relevant variables regarding the presence of parental psychopathology (e.g. parental ADHD) were not measured, only reasons for discontinuing treatment, with more participants in the blended condition finding the program too consuming/ elaborate. Given the sample involved in this study (e.g. parents of children diagnosed with ADHD, recruited from an outpatient mental health clinic) which is different from previous studies, it is possible that the clinical presentation is more severe, more comorbid conditions are present and there is a need for more intensive treatment (multimodal intervention involving both parents and teachers, combined treatment), tailored to different clinical comorbid presentations.

These preliminary results regarding the feasibility and efficacy of digital interventions for parents of children with ADHD call for further investigation. Despite the initial positive results, attendance and engagement are important factors; therefore, low-intensity, briefer Internet-delivered interventions are needed for this population. Varied outcomes of prior studies investigating online programs could be explained by the high heterogeneity between these programs and studies; specifically, they could be attributed to the diversity of the study participants involved (parents of children at risk for ADHD vs. of children diagnosed with ADHD), varying lengths of treatment and different therapist involvement in the program (face-to-face meetings or telephone contacts with research team members). Furthermore, existing programs were based on therapeutic protocols evaluated in clinical trials. Iterative development in online programs seems to be an important aspect of parents’ engagement, as while several programs simply adapted the traditional face-to-face delivery method to an online format, while others, such as the study of DuPaul,¹⁰ achieved high adherence by continuously refining the program through an iterative development process. In order to develop the online version of the program and to promote participants’ engagement, the authors revised an existing protocol in a 5-step iterative process.¹⁴ The development of the Promoting Engagement with ADHD Pre-Kindergartners (PEAK) program involved consultation of different stakeholders (e.g. parents, teachers, physicians, psychologists, social workers, special educators) and included multiple phases of testing of the revised versions of the intervention after feedback was received.

The present study

The aims of the present study were to investigate the feasibility, acceptability and usability of a co-designed Internet-delivered intervention developed for parents of children with ADHD. We also investigated the acceptability and user feedback with mental health specialists. While existing studies indicate that digital interventions could be used to deliver BPT, so far no study has been conducted with an East European population. All the existent research on online programs addressed to parents of children with ADHD has been conducted in high developed countries (e.g. the Netherlands,⁹ USA,¹⁰ New Zeeland¹¹). Furthermore, all these studies involved either face-to-face or telephone contacts with a specialist/ team member in order to increase adherence. It is unclear so far whether an Internet-delivered intervention with minimal therapist support is feasible for parents of children diagnosed with ADHD, from a developing country. It is known that treatment access is lower in developing countries, with pharmacological treatment being more accessible as compared to psychotherapy interventions.¹⁵ Therefore, an Internet-delivered intervention could reduce the gap in treatment access.

Method

Study design

A mixed-methods study was conducted in order to investigate the acceptability, usability and feasibility of ADHDCoach with data collected through a focus group with mental health specialists and an open trial with parents of children diagnosed with ADHD.

Participants

Participants were mental health specialists (child psychiatrists) and parents of children diagnosed with ADHD recruited from a university child psychiatry clinic from Cluj-Napoca, Romania. This is a medical center, where parents and children from both urban and rural areas around Cluj-Napoca access inpatient/ outpatient treatment, paid by insurance.

Inclusion and exclusion criteria

Mental health specialists were eligible if they worked with children diagnosed with ADHD.

Inclusion criteria for parents of the present study were: (a) having a child diagnosed with ADHD, (b) aged between 5 and 14 years, (c) having the ability to read and write in Romanian, (d) there was no change in the pharmacological treatment in the last 3 months and (e) having access to the Internet.

Exclusion criteria for parents: currently undergoing behavioral parent training for child ADHD or participating in other research investigating the efficacy of a parenting program, child's age was below 5 or above 14, child primary diagnosis was autism spectrum disorder.

Parents of children with ADHD (n = 29) were recruited between January 2022 and March 2022 from outpatients of a Child Psychiatry Clinic in Cluj-Napoca and were interviewed by a child psychiatrist to qualify for ADHD diagnosis.

Procedure

The study was approved by the Ethics Committee of Babeș-Bolyai University. Participants were recruited from the Child Psychiatry Clinic in Cluj-Napoca. Mental health specialists signed the informed consent and participated in a focus group with a research assistant at the Child Psychiatry Clinic from Cluj-Napoca. The duration of the focus group was around 60 min and the topic guide covered questions regarding specialists’ experience with digital mental health interventions (Internet-delivered interventions, mobile apps) in their current practice, their recommendations for these interventions to be used by their patients (children or adolescents with mental health problems, or specifically with ADHD, and to their parents), their attitudes of the usefulness of an Internet-delivered intervention in their current practice in the treatment of children diagnosed with ADHD, and the specific desired features recommended for this intervention (e.g. which characteristics this Internet-delivered intervention should have). The ADHDCoach program was projected on a screen, and the content of the modules was briefly described. Then, specialists answered several questions regarding what they liked and disliked about ADHDCoach, its ease of use and design, what functionalities they would consider eliminating, what enhancements should be implemented to develop and refine this intervention in order to address better patients’ needs, as well as identifying any potential barriers parents of children with ADHD might encounter when using this intervention. The focus group was audio recorded and transcribed. After the focus group ended, mental health specialists completed a questionnaire regarding their attitudes towards digital mental health interventions.

Parents were recruited from outpatients from the same child psychiatry clinic and invited to participate in this research during their regular meetings with a child psychiatrist. Parents signed informed consent and completed a screening assessment. After completing the assessment instruments, a research assistant contacted eligible participants by telephone and gave instructions to create an account on the secure online ADHDCoach platform. Parents were instructed to log-in, read the modules and complete the exercises to apply the contents described in the program. The duration of the intervention was four weeks, after which parents were invited to complete post-intervention assessments.

Measures

Demographic information

Basic information regarding parents’ age, gender, occupational status, number of children, child's age and gender was collected using a demographic questionnaire. Mental health specialists also completed a demographic questionnaire about their age, gender and number of years of experience in their profession.

Parents’ preferences about internet-delivered interventions

Parents were asked to answer the following question: “What would be, for you, the ideal way to participate in this program?” There were three options available, namely: (a) having access, via an online platform, to a set of multimedia materials and browsing them individually (self-help), (b) joining online group sessions with other parents who are experiencing similar difficulties or (c) having access to online materials and telephone contact to discuss difficulties encountered in applying the techniques learned in the program.

Feasibility and acceptability of the program for parents

The feasibility of the program was assessed through information regarding ADHDCoach usage. Data regarding participants’ activity on the platform (e.g. number of log-ins, number of modules read, homework exercises completed) was extracted from the platform.

The acceptability of ADHDCoach was assessed using a feedback questionnaire regarding ADHDCoach contents. Parents answered two open questions regarding what they liked and disliked on the online platform.

Program usability

We adapted the System Usability Scale (SUS).^16,17 The scale contains 10 items, rated from 1 (Strongly disagree) to 5 (Strongly Agree). Scores range between 0 and 100, higher scores indicating higher usability of the platform. Cronbach's alpha was good for the total scale (= .84).

User friendliness

We asked participants “How friendly did you find the platform?”, which was previously used in similar research on the efficacy of an Internet-delivered intervention.¹⁸ Answers are rated from 1 (Very unfriendly) to 10 (Extremely friendly), higher scores indicating higher program-perceived friendliness.

Satisfaction with the program

We measured satisfaction with the program via the Client Satisfaction Questionnaire-8.¹⁹ The questionnaire has eight items rated on a 4-point Likert scale (sample items: “How would you rate the quality of the treatment you have received?”, “If you were to seek help again, would you come back to this treatment?”). Scores range between 0 and 32, with higher scores indicating higher satisfaction with the Internet-delivered intervention. Cronbach's alpha was good for the total scale (= .81).

Working alliance

Parents rated their working alliance with the therapists assigned on the platform using the Working alliance Inventory for guided Internet interventions (WAI-I)²⁰. This scale consists of 12 items, rated on a 5-point Likert scale from 1 (Never) to 5 (Always). Sample items: “With the online program, it has become clearer to me how I can change”, “The psychologist who supports me in the online program is really interested in my well-being”. Scores can be computed for the total scale or for two subscales, namely Bond with therapist and Task and goal agreement with program. Total scores range between 1 and 5, higher scores indicate higher working alliance. Cronbach's alpha for the total scale (= .95) and for the Task and goal agreement with program subscale (= .94) and adequate for Bond with therapist subscale (= .85).

Mental health specialists’ therapy attitudes

Mental health specialists completed the Electronic Therapy Attitudes and Process Questionnaire (e-TAP-T)²¹ at the end of the focus group. This is a 12-item questionnaire that assesses factors related to professionals’ engagement with digital interventions for clients’ mental health problems. Items are rated on a 7-point Likert scale (1—Strongly Disagree, 7—Strongly Agree; items 1, 5, 9 are rated from −3 to +3 and are rescaled from 1 to 7 as well), higher scores indicating higher likelihood of engaging digital interventions in treatment. The questionnaire was developed based on the Theory of Planned Behavior.²² Scores can be computed for four subscales, namely: attitudes (positive or negative attitudes regarding engaging digital interventions; sample items: “I find the use of digital interventions for clients’ mental health to be: unpleasant/ pleasant ”, “I find the use of digital interventions for clients’ mental health to be: harmful/ beneficial”), perceived behavioral control (self-efficacy beliefs; sample items: “I am confident in my ability to use digital interventions with clients”, “I possess the required technical knowledge to use digital interventions for my clients’ mental health”), subjective norms (impressions of others’ viewpoints; sample items: “My peers would support my use of digital interventions with my clients”, “My colleagues would support my use of digital interventions with clients”) and behavioral intention (intention to engage in digital interventions; sample items: “I will learn more about digital interventions”, “I intend to check with my clients whether they would like to use digital interventions to address their mental health concerns”). Total scores for each subscale are averaged (scores range for each subscale between 1 and 7). Cronbach's alpha for the total scale was .86.

Intervention

The Internet-delivered intervention: ADHDCoach

The Internet-delivered intervention, called ADHDCoach, was developed by the first author, based on an existing Behavioral Parent Training protocol, an intervention that was previously tested in a randomized controlled trial with Romanian children.²³ ADHDCoach adhered to the same outline as the original protocol contents (modules 1–5), however, interactive materials (videos depicting a specialist presenting the contents of the module, videos depicting child-parent interactions, quizzes) were devised to present the information contained in the original protocol. While the original protocol is delivered via 16 weeks, in our Internet-delivered intervention we compressed the BPT section given that we aimed to develop a briefer version of it. In order to develop modules that could be used for tailoring the program in order to address parental distress, or comorbid child mental health problems (e.g. anxiety disorders, depressive disorders), the contents for parent modules were developed according to Rational Emotive Behavior Therapy (REBT)²⁴. According to REBT theory, psychological distress emerges as a consequence of irrational beliefs, and not as a direct consequence of external events.

The intervention consisted of nine modules (Figure 1) delivered over 4 weeks (see Table 1 for the key aspects covered in each module). ADHDCoach is available in Romanian, the program was designed to be used independently by the parent.

Figure 1.

Screenshot of the structure of the nine modules from the Internet-delivered intervention.

Table 1.

Module contents.

Module name	Strategies	Module content
Module 1. Introduction	Psychoeducation regarding the program	Introduction to program Set-up expectancies regarding parental involvement in treatment Formulate SMART program goals
Module 2. What is ADHD?	Psychoeducation regarding ADHD	Present ADHD symptoms Discuss the causes of ADHD Combat common myths regarding ADHD
Module 3. Evidence-based interventions in ADHD	Psychoeducation regarding treatments for ADHD	Discuss what evidence-based treatments are available for ADHD
Module 4. Cultivating a positive relationship with your child	Journal of positive experiences Plan a fun time Play	Learn about the ballance between positive and negative experiences Present ideas to improve parent-child relationship
Module 5. Behavioral parent training	Behavioral techniques (Time-Out, reward systems)	Present the distinction between requests and rules Learn how to implement a reward plan
Module 6. Comorbid problems associated with ADHD	Psychoeducation regarding internalizing and externalizing comorbid problems in ADHD	Validate child's emotions Rational thinking patterns Behavioral activation with child
Module 7. Parental distress	Identifying distress Cognitive restructuring	Learn the relationship between irrational beliefs and distress Discuss modalities to dispute irrational thinking and help the parent develop rational thinking Practice behavioral activation
Module 8. Family relationships	Problem solving	Learn a problem solving approach Discuss communication techniques to inform friends and family members on ADHD Maintain a close relationship with other siblings
Module 9. Prepare for future	Maintenance of treatment gains	Review techniques learned during this program Establish long-term goals to continue progress Plan for future difficult situations where the child may feel distressed or exhibit problematic behavior

A structured format was used for each module, with the following components of each: the session's objectives, the primary topics of the module, a summary and homework assignments. In each module, there is written content (Figure 2), as well as audio or video content.

Figure 2.

Screenshot of an exercise from module 7 of ADHDCoach.

Each module included 3–5-min animations with parent–child dyads describing typical problems that appear in families of children diagnosed with ADHD, with specific instructions on how to manage such situations. Furthermore, each module contained 2–3 min of animation with a specialist discussing parenting skills. Parents were invited to progress through the modules at their own pace, with 7 days before new content was available.

The Internet-delivered intervention was guided by two psychotherapists under supervision who were trained in Cognitive Behavioral Therapy, and trained by the first author on the treatment protocol. They offered feedback on module worksheets and homework assignments, as well as providing help and responding to parents’ inquiries. All communication between therapists and parents was done online via a secure online platform. When parents completed homework assignments as well as when they asked questions, the therapist received notifications to log-in to the secure platform and provide answers.

Statistical analysis

Qualitative data obtained from the focus group was analyzed through Thematic analysis. Two independent researchers closely examined the transcripts and key themes were identified. Quantitative data were analyzed using SPSS version 26 (IBM Corp). Descriptive statistics (frequencies, means and standard deviations) are reported for participants’ characteristics, adherence to program, user-friendliness and usability. Spearman correlation coefficients were calculated for the relationship between working alliance and participants’ satisfaction with the Internet-delivered intervention.

Results

At baseline, 29 parents showed interest in the program. Of the initial number of parents interested, five parents did not complete the baseline assessments. The total number of eligible participants was 24 parents of children diagnosed with ADHD (M_age = 41.50, SD = 8.98, 87.5% mothers, M_{child age} = 10.08, SD = 3.30, 79.2% boys). Most participating parents were married (n = 19, 79.2%), full-time employed (n = 19, 79.2%), had an urban residency (n = 18, 75%) and were educated (n = 16, 66.6% had a university degree). From the current sample, 17 children (n = 70.83%) were undergoing pharmacological treatment. All children had a primary diagnosis of ADHD, 12 children (50%) also presented comorbid conditions (e.g. 7 with oppositional defiant disorder, 1 with anxiety disorders, 1 with major depressive disorder, 3 with both oppositional defiant disorder and anxiety disorders).

Mental health specialists (n = 16) were all psychiatrists in the specialty of pediatric psychiatry, aged between 26 and 42 (M_age = 29.25, SD = 3.75) with work experience of 1–15 years.

Parents’ ratings of ADHDCoach feasibility, acceptability and usability

Participants completed a mean number of 5 modules (SD = 2.26) of the nine available modules. The number of modules read by participants ranged between 2 and 9. A total of 3 parents (12.5%) completed all 9 modules, 3 parents (12.5%) completed 2 modules, 5 parents (20.83%) completed 3 modules, 2 parents (8.33%) completed 4 modules, 11 (45.83%) completed more than 5 modules. Regarding homework assigned, 6 parents (25%) did not complete any homework and 18 parents (75%) completed at least one homework assignment during the 4 weeks.

On the Usability scale, the mean was 75.79, SD = 13.59, which according to the Sauro–Lewis curved grading scale represents an acceptable product (grade B).²⁵

Parents reported that the intervention was user-friendly (M = 9.15, SD = 1.11). Satisfaction with the intervention was high (M = 25.42, SD = 4.11).

Parents’ feedback on the program

Overall, parents rated the program as helpful and indicated the most useful function to be the informative one, namely the contents. They disliked repeated psychological assessments and indicated that shorter psychological instruments would be useful. Also, parents indicated that more tailoring based on their home situations would be useful. Several themes emerged from parents’ responses regarding the aspects they liked or disliked about ADHDCoach. Specifically, parents most like the content (practical information presented in an interactive manner), structure and ease of access of ADHDCoach. On the other hand, parents disliked the psychological assessment section, the deadlines for homework completion and needed more detailed information and everyday examples.

Parents’ preferences regarding their involvement in ADHDCoach

Regarding parents’ preferences about the ideal implication in the present program, nine parents selected the first option, namely having access, via an online platform, to a set of multimedia materials and browsing them individually (self-help). Only two parents chose the second option, respectively, to join online group sessions with other parents who are experiencing similar difficulties, while 10 parents selected the third option, to have access to online materials and telephone contact to discuss difficulties encountered in applying the techniques learned in the program.

Working alliance

Participants reported a high working alliance with their therapists based on the total scale (M = 3.66, SD = 0.94) and for the two subscales scores, namely bond with the supporting therapist (M = 3.64, SD = 1.01) and tasks and goals agreement with the program (M = 3.67, SD = 0.96). Working alliance was significantly associated with participants’ satisfaction with the Internet-delivered intervention, the total score in working alliance (r_s = .71, p < .001), as well as the two dimensions, bond with the supporting therapist (r_s = .83, p < .001) and tasks and goals agreement with the program (r_s = .54, p = .017) were related to parents’ satisfaction with ADHDCoach.

Focus group with specialists

Most specialists mentioned that they were using digital mental health interventions in their current practice and gave examples of mood monitoring apps they particularly recommend for adolescents. They considered useful in their activity with children with ADHD an Internet-delivered intervention. The key themes that emerged during the focus group with specialists were program design, content and integration with other functions.

Key theme 1: design

Mental health professionals considered the overall design simple and the interface easy to use. They recommended using bold letters in order to emphasize the most important information presented in the program and less content on a page which could be affected by parents’ attention spans.

Key theme 2: content

In terms of content, most of the participants agreed that the program is complex and includes all the components of an evidence-based behavioral parent training program. They suggested further development of videos that illustrate techniques. In several modules, the amount of material was deemed excessive, and it was proposed that the too-long parts be divided up.

Key theme 3: integration and further developments

They suggested adding a forum section, where parents could discuss the difficulties they face, the implementation of several techniques from the program and integrating the program within mental health care.

Mental health specialists’ engagement factors with digital interventions

Specialists’ scores for the e-TAP-T scale for attitudes (M = 6.54, SD = 0.64), perceived behavioral control (M = 6.41, SD = 0.69), subjective norms (M = 6.31, SD = 0.56) and behavioral intention (M = 6.22, SD = 0.68) were high.

Discussion

Child ADHD is a significant burden for parents, children and society. Mixed evidence regarding online behavioral parenting programs calls for further developments in such interventions that could overcome gaps in treatment. Through this study, we aim to present the development, feasibility and usability of ADHDCoach, an Internet-delivered intervention for parents of children with ADHD. To our knowledge, this is the first Internet-delivered intervention developed based on the REBT approach with a sample of parents of children diagnosed with ADHD.

Our results indicated that the program is feasible given the number of completed modules and the high satisfaction of the participants with the intervention. Participants perceived ADHDCoach to be friendly and reported acceptable system usability. According to parents’ feedback, the most appreciated components were the contents of the intervention delivered in a structured and interactive manner, and the least appreciated were the psychological assessments completed online. According to specialists’ feedback, several changes need to be considered in order to improve access to remote areas and to parents with a low levels of education.

The working alliance was high, similar to the previous research.²⁰ As in face-to-face therapy, working alliance is an important predictor of treatment outcome in Internet-delivered and blended interventions both at postintervention and at follow-up assessments.^26,27 Results of a study conducted with patients with panic disorder that received an Internet-delivered intervention, indicated that the two components of the working alliance could have different contributions to the therapy gains. Namely, alliance with a therapist could enhance treatment outcomes, while agreement on tasks and goals could improve adherence.²⁸ Therefore, it is highly important to investigate parents’ perspectives on the alliance with the therapist assigned in the program, as well as their agreement on tasks and goals within the ADHDCoach program, and to find new modalities to improve the working alliance. Working alliance was significantly related to participants’ satisfaction with ADHDCoach, with both dimensions, bond with therapist and tasks and goals agreement with the program, related to satisfaction, which is similar to previous research conducted with guided Internet-delivered interventions.^20,29 Interestingly, we found a different association between the two subscales and participants’ satisfaction, as while in previous research²⁰ there was a stronger association between tasks and goals agreement with the program and satisfaction, in our study, bond with therapist was stronger related to parents’ satisfaction.

Mental health specialists involved in this study provided recommendations regarding ADHDCoach design, contents covered and further integrations. Their attitudes about Internet-delivered interventions were positive, with perceived behavioral control and subjective norms similar to previous research, however, behavioral intention scores were higher than in previous studies.²¹ These results could indicate that mental health specialists need to integrate digital interventions into their care which could overcome barriers in monitoring treatment response of patients over time, providing them with access to evidence-based interventions at low costs, since in many countries mental health services are not covered by health insurances.

Our results need to be interpreted taking into consideration several limitations. Regarding the specialists involved in developing ADHDCoach, we included only psychiatrists working with children with ADHD. Involving interdisciplinary professionals, such as neuropediatricians, psychotherapists, school counselors, teachers and other stakeholders could be a more extensive approach to develop and refine this digital tool for child ADHD.

A major limitation of the present study is the lack of blind assessments, as no assessments were provided by teachers or clinicians. We relied on self-reports only, which could bias our results. Also, the small sample size is another important limit that might influence our findings. The brief duration of the program could be an important limitation for those parents who did not have enough time to access all the modules, or for parents with ADHD symptoms. Another limitation is related to the lack of a follow-up assessment in order to investigate potential changes in ADHD symptoms or parent-related variables after the intervention. The absence of a control group is another limitation of the present study; the presence of an active control group (e.g. traditional, face-to-face BPT) could have been informative regarding the preliminary efficacy of ADHDCoach in comparison with other interventions. A larger randomized controlled trial conducted to investigate the efficacy of ADHDCoach could address these limitations.

The development of an Internet-delivered intervention for parents of children with ADHD is an innovative approach to deliver evidence-based treatments for parents worldwide. Our results are in line with other research indicating that online behavioral parenting interventions are promising alternatives for ADHD in children,¹⁰ however, attrition is an important limitation of such programs. Given that only a small percentage of parents completed all the modules assigned within the designated timeframe and given the large percentage of parents who did not complete homework assignments, an important endeavor of future research is to find new modalities to engage parents of children diagnosed with ADHD actively in such interventions. Research on this topic is only at the beginning, with results of a randomized controlled trial conducted with a similar population⁹ indicating that parents invest less effort in the blended condition (online BPT + face-to-face contacts) as compared to control (face-to-face condition). As our intervention did not include face-to-face contact, it is possible that parents’ efforts were reduced, as reflected in their homework compliance. Barriers to engagement can be related to multiple factors, such as parents’ attitudes towards digital mental health interventions, parents’ attitudes regarding child's diagnosis, or parents’ expectations regarding their own involvement in treatment,³ logistic barriers (lack of time, presence of distractions at home like household chores and child care, the presence of other children in the house³⁰) or parent psychopathology³¹ (parent ADHD, parent depression). Another barrier to the adoption of digital mental health interventions is the lack of personalization.³²

Future studies could benefit from qualitative assessments (e.g. interviews conducted with parents) in order to investigate factors related to the low compliance with homework. Future research should investigate different modalities to engage parents in such interventions, such as: using motivational interviewing techniques,³³ blending Internet-delivered interventions with face-to-face meetings,⁹ using telephone contacts^11,12 or scheduling video conferences with parents³⁴ in order to increase their engagement. The literature suggests various strategies for increasing engagement in digital mental health interventions such as personalizing feedback, providing e-coaching, offering social forums for participants to connect with others, sending reminders, incorporating gamification elements, allowing for flexible scheduling and simplifying the user experience.³⁵ ADHDCoach has already incorporated several strategies for engagement (e.g. providing guidance and feedback from a coach, sending notifications when new modules are allocated and reminders to complete assessments). However, other strategies could also be implemented to enhance participants’ engagement, such as adding a forum section (as suggested by the participating professionals) where parents can connect and share challenges with other parents participating in the program. Additionally, integrating a mobile app,³⁶ could also be used to support homework assignments.

Also, Internet-delivered interventions informed by network analyses conducted with parents of children with mental health difficulties³⁷ could be particularly important in order to provide individually tailored interventions for families of children with ADHD. According to the network approach to psychopathology, disorders are not entities, but are being composed of symptoms that interact with one another.³⁸ Network analysis could indicate which symptoms improve after the ADHDCoach intervention and how they are directly and indirectly connected. Using network analysis one can assess whether the symptoms (nodes) with the highest centralities are different between groups (e.g. boys and girls, parents with high vs. low distress, children with high levels of comorbid internalizing vs. externalizing problems). This can be achieved by performing network invariance tests. As a result of this analysis, it is possible to determine whether the primary intervention targets for different groups should be similar or distinct. Future studies should investigate the efficacy of ADHDCoach in randomized controlled trials with larger samples of participants, include blind assessments, as well as long-term follow-up. ADHDCoach can be further tested in different formats, such as blended with face-to-face interventions, combined with a child-component (adding specific online modules addressed to children or incorporating virtual reality in order to improve children's cognitive deficits³⁹).

Conclusion

Through this study, we aimed to present the development, feasibility and usability of a novel Internet-delivered intervention developed for parents of children with ADHD in co-design with parents and mental health professionals. Users’ feedback and the results of this study will be incorporated, and ADHDCoach will be tested in a randomized clinical trial. Further research should also consider including a larger sample size, employing clinician assessments and, nevertheless, investigating both short-term and long-term effects of the intervention, as well as mechanisms of change.

Footnotes

Acknowledgment

The authors would like to thank all the participants who took part in this research.

Contributorship

CRP and DD researched literature and conceived the study. AD and EP were involved in protocol development. AN and RȘ were involved in patient recruitment and data collection. CRP wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

The Ethics Committee of Babeș-Bolyai University approved this study.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a grant of the Romanian Ministry of Education and Research, CNCS–UEFISCDI, project number PN-III-P1-1.1-PD-2019-1191, within PNCDI III awarded to Dr Costina-Ruxandra Păsărelu.

Guarantor

CRP.

Trial registration

Not applicable.

ORCID iDs

Costina-Ruxandra Păsărelu

Elena Predescu

References

Dobrean

Păsărelu

Döpfner

. Varieties of psychotherapy for attention-deficit hyperactivity disorder: an evidence-based evaluation. In: David D, Lynn SJ, and Montgomery GH (eds) Evidence-based psychotherapy: the state of the science and practice. New York, NY: Wiley-Blackwell, 2018, pp. 435–463.

National Institute for Health and Care Excellence. https://www.nice.org.uk/guidance/ng87/resources/attention-deficit-hyperactivity-disorder-diagnosis-and-management-pdf-1837699732933 (accessed 8 February 2022).

Baweja

Soutullo

Waxmonsky

. Review of barriers and interventions to promote treatment engagement for pediatric attention deficit hyperactivity disorder care. World J Psychiatry 2021; 11: 1206–1227.

Chacko

Jensen

Lowry

, et al. Engagement in behavioral parent training: review of the literature and implications for practice. Clin Child Fam Psychol Rev 2016; 19: 204–215.

Păsărelu

Andersson

Bergman Nordgren

, et al. Internet-delivered transdiagnostic and tailored cognitive behavioral therapy for anxiety and depression: a systematic review and meta-analysis of randomized controlled trials. Cogn Behav Ther 2017; 46: 1–28.

Florean

Dobrean

Păsărelu

, et al. The efficacy of internet-based parenting programs for children and adolescents with behavior problems: a meta-analysis of randomized clinical trials. Clin Child Fam Psychol Rev 2020; 23: 510–528.

Păsărelu

Andersson

Dobrean

. Attention-deficit/ hyperactivity disorder mobile apps: a systematic review. Int J Med Inf 2020; 138: 104133.

Shou

Xiu

, et al. Efficacy of online intervention for ADHD: a meta-analysis and systematic review. Front Psychol 2022; 13: 854810.

Breider

de Bildt

Nauta

, et al. Self-directed or therapist-led parent training for children with attention deficit hyperactivity disorder? A randomized controlled non-inferiority pilot trial. Internet Interv 2019; 18: 100262.

10.

DuPaul

Kern

Belk

, et al. Face-to-Face versus online behavioral parent training for young children at risk for ADHD: treatment engagement and outcomes. Journal of Clinical Child & Adolescent Psychology 2018; 47: S369–S383.

11.

Franke

Keown

Sanders

. An RCT of an online parenting program for parents of preschool-aged children with ADHD symptoms. J Atten Disord 2020; 24: 1716–1726.

12.

Döpfner

Wähnke

Klemp

M-T

, et al. Efficacy of web-assisted self-help for parents of children with ADHD (WASH) – a three-arm randomized trial under field/routine care conditions in Germany. BMC Psychiatry 2020; 20: 76.

13.

Dekkers

Hornstra

van der Oord

, et al.

Meta-analysis: Which components of parent training work for children with attention-deficit/hyperactivity disorder?

J Am Acad Child Adolesc Psychiatry 2021; 61: 478–494.

14.

DuPaul

Kern

Belk

, et al. Promoting parent engagement in behavioral intervention for young children with ADHD: iterative treatment development. Topics Early Child Spec Educ 2018; 38: 42–53.

15.

Pipe

Ravindran

Paric

, et al. Treatments for child and adolescent attention deficit hyperactivity disorder in low and middle-income countries: a narrative review. Asian J Psychiatr 2022; 76: 103232.

16.

Bangor

Kortum

Miller

. An empirical evaluation of the system usability scale. International Journal of Human–Computer Interaction 2008; 24: 574–594.

17.

Brooke

. SUS: A ‘quick and dirty’ usability scale. In: Jordan PW, Thomas B, McClelland IL, and Weerdmeester BA (Eds) Usability evaluation in industry. London: Taylor & Francis, 1996, pp. 189–194.

18.

Păsărelu

C-R

Dobrean

Andersson

, et al. Feasibility and clinical utility of a transdiagnostic internet-delivered rational emotive and behavioral intervention for adolescents with anxiety and depressive disorders. Internet Interv 2021; 26: 100479.

19.

Attkisson

Zwick

. The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann 1982; 5: 233–237.

20.

Gómez Penedo

Berger

grosse Holtforth

, et al. The working alliance inventory for guided internet interventions (WAI-I). J Clin Psychol 2020; 76: 973–986.

21.

Clough

Rowland

Casey

. Development of the eTAP-T: a measure of mental health professionals’ attitudes and process towards e-interventions. Internet Interv 2019; 18: 100288.

22.

Ajzen

. The theory of planned behavior. Organ Behav Hum Decis Process 1991; 50: 179–211.

23.

David

Dobrean

Păsărelu

, et al. Psychotherapy, atomoxetine or Both? Preliminary evidence from a comparative study of three types of treatment for attention-deficit/hyperactivity disorder in children. Cogn Ther Res 2021; 45: 149–165.

24.

Ellis

. Reason and emotion in psychotherapy: revised and updated. New York, NY: Birch Lane, 1994.

25.

Lewis

. Measuring perceived usability: the CSUQ, SUS, and UMUX. International Journal of Human–Computer Interaction 2018; 34: 1148–1156.

26.

Luo

Bugatti

Molina

, et al. Conceptual invariance, trajectories, and outcome associations of working alliance in unguided and guided internet-based psychological interventions: secondary analysis of a randomized controlled trial. JMIR Ment Health 2022; 9: e35496.

27.

Vernmark

Hesser

Topooco

, et al. Working alliance as a predictor of change in depression during blended cognitive behaviour therapy. Cogn Behav Ther 2019; 48: 285–299.

28.

Zalaznik

Strauss

Halaj

, et al. Patient alliance with the program predicts treatment outcomes Whereas alliance with the therapist predicts adherence in internet-based therapy for panic disorder. Psychother Res 2021; 31: 1022–1035.

29.

Gómez Penedo

Babl

grosse Holtforth

, et al. The association of therapeutic alliance with long-term outcome in a guided internet intervention for depression: secondary analysis from a randomized control trial. J Med Internet Res 2020; 22: e15824.

30.

Tarver

Daley

Sayal

. A self-help version of the new forest parenting programme for parents of children with attention deficit hyperactivity disorder: a qualitative study of parent views and acceptability. Child Adolesc Ment Health 2022; 27: 215–222.

31.

Friedman

Dvorsky

McBurnett

, et al. Do Parents’ ADHD symptoms affect treatment for Their children? The impact of parental ADHD on adherence to behavioral parent training for childhood ADHD. J Abnorm Child Psychol 2020; 48: 1425–1437.

32.

Borghouts

Eikey

Mark

, et al. Barriers to and facilitators of user engagement with digital mental health interventions: systematic review. J Med Internet Res 2021; 23: e24387.

33.

Sibley

Graziano

Kuriyan

, et al. Parent-Teen behavior therapy + motivational interviewing for adolescents with ADHD. J Consult Clin Psychol 2016; 84: 699–712.

34.

Diaz-Stransky

Rowley

Zecher

, et al. Tantrum tool: development and open pilot study of online parent training for irritability and disruptive behavior. J Child Adolesc Psychopharmacol 2020; 30: 558–566.

35.

Saleem

Kühne

Santis

, et al. Understanding engagement strategies in digital interventions for mental health promotion: scoping review. JMIR Ment Health 2021; 8: e30000.

36.

Păsărelu

C-R

Kertesz

Dobrean

. The development and usability of a Mobile app for parents of children with ADHD. Children 2023; 10: 164.

37.

Păsărelu

C-R

Dobrean

Florean

, et al. Parental stress and child mental health: a network analysis of Romanian parents. Curr Psychol 2022. Epub ahead of print 4 August 2022. DOI: 10.1007/s12144-022-03520-1.

38.

Borsboom

Deserno

Rhemtulla

, et al. Network analysis of multivariate data in psychological science. Nat Rev Methods Primers 2021; 1: 1–18.

39.

Corrigan

Păsărelu

C-R

, & Voinescu

. Immersive virtual reality for improving cognitive deficits in children with ADHD: a systematic review and meta-analysis. Virtual Reality 2023; 5: 304.