Development and usability testing of an internet-based self-help program integrating acceptance and commitment therapy and behavioral activation intervention for ethnic minority young adults with alcohol use disorder: Usability study

Abstract

Introduction

Alcohol use disorder (AUD) is a severe medical problem that affects ethnic minority young adults. An internet-based self-help program integrating acceptance and commitment therapy (ACT) and behavioral activation (BA) was found promising. However, such integration has never been applied to people with AUD. Therefore, developing and testing this integration was warranted.

Objectives

This study aimed to develop and test an internet-based self-help program integrating ACT and BA among ethnic minority young adults with AUD.

Method

Four distinct phases were employed, including preparing self-help intervention content, expert panel review, developing an online intervention using Qualtrics, and usability testing. A total of 10 ethnic minority young adults with AUD were recruited for the usability test. Descriptive statistics and a one-sample t-test were employed to determine usability.

Result

A total of 10 participants were invited to participate in the study, with a completion rate of 90%. The overall usability score was 74.72 (SD = 4.75). About 88.9% of participants found the intervention helpful, easy to navigate, and clear content and instructions. Participants spent about 10–30 minutes per module. Almost all participants received one reminder per module for completion. However, three participants experienced device incompatibility, internet interruptions, and login failures.

Conclusion

A new internet-based self-help ACT-BA intervention was developed and tested. The findings showed an excellent usability score. Future interventions will consider simple plain language over technical terms and incorporate withdrawal symptom management. Besides, conducting a brief session during recruitment to guide participants and sending a reminder, especially on holidays and weekends, is warranted.

Keywords

acceptance and commitment therapy alcohol use disorder behavioral activation ethnic minority young adults usability testing

Introduction

AUD is a significant global health concern, which is particularly prevalent in developed countries due to the affordability, accessibility, and aggressive marketing of alcohol.¹ Although the prevalence of AUD varies across different demographics, including age, race, gender, and culture, all populations are at risk of developing the disorder.^2–4 A US epidemiological study highlights a rapid rise in AUD prevalence from 8.5% to 17.7% in the total population between 2001-2002 and 2012-2013. During this period, a significant increase was observed among ethnic minorities (rose by 92.8% in Black individuals, 51.9% in Hispanic individuals, 83.7% in women, and 200% for poor people).⁵ Similarly, a study in Hong Kong showed the overall prevalence of AUD among ethnic minorities was 19.8%, with 7.9%, 5.4%, and 6.4% for mild, moderate, and severe AUD, respectively.⁶ Recent studies also indicate that young ethnic minority populations are especially vulnerable to substance use, including alcohol, due to various reasons, such as relaxation, socialization, mood enhancement, stress relief, and coping with negative emotions.^2,7,8 Likewise, these populations often experience discrimination, stigma, and marginalization in areas such as employment, education, and social activities, leading to acculturation stress, depression, isolation, and life dissatisfaction,^9–11 which places them at risk of AUD.¹² Importantly, a recent study found that only 4% of ethnic minority college students received treatment for AUD, potentially due to a lack of culturally tailored treatments, isolation, work burden, and language barriers.¹³

In this regard, our research team has done a number of studies to address AUD in ethnic minority young adults in Hong Kong.^14,15 Our cross-sectional study identified male gender and the age range of 18 -35 as contributing to AUD in this population group. Likewise, despite acculturation leading to challenges to ethnic minorities, it is a golden opportunity for alcohol abstinence, given that the ethnic minorities are geographically away from the parental influence, which was confirmed to be one of the significant factors contributing to AUD.^16,17 Besides, the experience of discrimination and avoidance coping behaviours were shown to be significant factors contributing to AUD.¹⁸ These highlight that the future of interventions for AUD should address these two components, thus assisting ethnic minorities with AUD to achieve alcohol abstinence.

A comprehensive review of the literature indicated that acceptance and commitment therapy (ACT) may be a viable treatment option for ethnic minority young adults with AUD. ACT, as a third wave of cognitive behavioral therapies, aims to assist individuals to achieve a meaningful life that aligns with their life goals through accepting inevitable pain and suffering.^19–21 This intervention directly addresses the inevitable nature of discrimination and the use of avoidance coping behaviours in ethnic minorities with AUD. In fact, a systematic review and meta-analysis have already confirmed the effectiveness of ACT for addictive behaviours, despite only two included studies focusing on alcohol use.¹⁴ Another comprehensive literature review regarding the effectiveness of ACT on AUD also suggested that it was a viable treatment option.^22–27 Although ACT is a viable treatment option for AUD, a comprehensive literature review highlighted a substantial problem with participants’ engagement and adherence to the intervention.^26,28,29 In this review, many participants exhibited low engagement and adherence to the value-based activities that guide them to commit actions toward a meaningful life.^26,28 The low engagement and adherence may arise from a failure to identify enjoyable, rewarding, and positively reinforcing activities. Apart from the engagement issue, ACT primarily focuses on internal experiences such as negative emotions, thoughts, sensations, and memories, with insufficient attention to overt behaviors. Therefore, individuals may not have sufficient skills to manage the high-risk situations for drinking, notwithstanding that they understand that they should not use drinking as a coping method, resulting in high likelihoods of relapse and limited behavioural change. Besides, ACT lacks mechanisms to guide individuals in self-monitoring their daily activities and scheduling alternatives to replace drinking in their lives. However, these are critical challenges for ethnic minorities with AUD, as our previous qualitative study revealed that a substantial number of ethnic minorities with AUD attempted various coping strategies in quitting, but their engagement in alcohol-free activities remained a significant problem because they had no structured daily planning and monitoring skills, and experienced difficulties in identifying and engaging with enjoyable alcohol-free activities. As a result, participants’ efforts to quit were unsuccessful, although some managed to quit briefly; they eventually relapsed. Given the above, new strategies are needed to be incorporated into ACT to address these limitations, with an aim to enhance participants’ engagement and adherence to the intervention.

Behavioural activation (BA) may be a possible method to overcome the aforementioned limitations. BA is a concept from behaviourism that states behaviour can directly affect emotions, for better or worse.^30,31 Particularly, BA is guided by operant conditioning, which states that participants’ behavior and mood are mainly maintained through their environment.³² When they experience barriers like stress, craving, stigma, and discrimination, they often withdraw from the environment, leading to loss of reinforcement.³³ Regarding AUD, alcohol often becomes the primary negative reinforcer.³³ In this regard, BA breaks the cycle of avoidance through enhancing alternative alcohol free, reinforcing, and rewarding activities that provide positive reinforcement. Hence, individuals can put themselves in a situation that is likely to foster positive emotions. For example, a positive event such as listening to uplifting music can lead to happiness, and vice versa. BA is a widely recognized and theoretically sound approach that has been applied to treat depression.^34,35 Its application has been extended to substance use disorders (SUD) via providing rewarding experiences in daily life different from substance use, thus achieving abstinence and preventing relapse.^36–38 Evidence suggests that BA is a promising intervention for SUD as a standalone treatment or being integrated with other therapeutic modalities. The emphasis on rewarding experiences in BA exactly addresses the low engagement and adherence of ACT due to the failure of identifying enjoyable, rewarding, and positively reinforcing activities in the intervention process. In addition, BA involves two major techniques, which are self-monitoring daily activities and scheduling, allowing individuals to monitor their usual daily activities and alcohol use, and develop a plan to schedule their identified alcohol-free rewarding activities to replace drinking in their lives. Because of the distinctive benefits of BA, it appears to be a treatment component complementary to the shortcomings of ACT, thus enhancing the intervention effectiveness if they both are integrated into the treatment protocol. Also, behavioural activation is a brief intervention (i.e., simple, cost-effective, and short-term) that can be delivered within 4 weeks, with a total duration of less than 90 minutes,³⁹ which will not significantly prolong the duration of each intervention session, ensuring the participants’ adherence and engagement. In sum, ACT simply encourages an individual to ACT, which includes A ccepting the reactions and being present, C hoosing a valued direction, and T ake actions, while BA motivates an individual to take real actions. In fact, two studies have already integrated BA into ACT to manage depression and psychotic features.^37,40 The results were promising, which supported that the integration is not only theoretically sound, but also feasible and importantly can translate into observable benefits.

Concerning the mode of delivery of the intervention, an online mode may be a more favourable option for ethnic minorities with AUD than face-to-face formats due to its improved accessibility and availability, reduced stigma, and non-interference with daily activities and work.^41–43 This idea is further supported by the widespread international development of websites and online technology for screening and initial counselling for alcohol use issues in the past decade, especially after the COVID-19 pandemic.^44,45 A review of 41 articles involving more than 50000 participants has shown that web-based interventions were more effective in reducing alcohol consumption as compared to non-web-based interventions or no intervention in adults, notwithstanding that none of the included articles were from Eastern countries, including China.⁴⁶

Although BA has been integrated into ACT to manage different conditions, mainly delivered in a face-to-face format,³⁷ which is an important omission in the literature, given the benefits of web-based interventions to addictive behaviours, no such integration was applied among ethnic minority young adults with AUD. To address this gap in existing literature, our research team aimed to develop an internet-based self-help program integrating ACT and BA intervention for young adult ethnic minorities with AUD in Hong Kong. Before a large-scale RCT, evaluating this new intervention protocol with the end users is crucial to improve and revise the intervention. In this regard, usability testing is a key component of the intervention development. Usability testing can improve participant engagement by tailoring navigation and content to the needs and preferences of participants. Therefore, this study was a proof of concept which aimed to design an internet-based self-help ACT-BA intervention and test its usability among Hong Kong ethnic minority young adults with AUD.

Methods

This study was reported in accordance with the GUIDE reporting checklist. The completed checklist is provided as a Supplementary file 1.

Procedures for intervention development

The development of the intervention for ethnic minority young adults with AUD involved four distinct phases before the actual implementation of the protocol. These phases were: (1) preparation of self-help intervention content; (2) expert panel review; (3) development of the online intervention using the Qualtrics platform; and (4) usability testing. The usability test was conducted cross-sectionally, using direct observation of application navigation tasks and a quantitative self-administered questionnaire to identify areas of improvement.

Phases of intervention development

Phase 1: Preparation of self-help iACT-BA intervention content

Modules and content of interventions

Two authors (GMB and KYH) have developed the self-help iACT-BA intervention protocol. The intervention content was adapted from the ACT manual,^47,48 self-help protocols,²⁸ and behavioral activation treatment for substance use and depression.^37,38

The intervention components were drawn from the ACT manual, which includes six core ACT components^47,48: (1) Acceptance; (2) Cognitive delusion; (3) Being present; (4) Self as context; (5) Values; and (6) Committed action. Additionally, Behavioral Activation (BA) components such as daily activity monitoring, activity scheduling, daily plans, and post-treatment plans were incorporated. The content was validated, and the maximum number of sessions was determined based on available literature. Participants or end users of the session were ethnic minority young adults with AUD. Each participant engaged in six sessions, with one module per week over six weeks. The integrated intervention (Self-help iACT-BA) was delivered in the following sequence: (1) introduction, confronting agenda, and daily monitoring; (2) values and activity scheduling; (3) acceptance and diffusion; (4) being present and observing self; (5) daily plans and committed action; (6) relapse prevention and plan. The participant accessed modules every week and completed assignments within the designated time.

Session 1: Introduction, confronting the agenda, and daily monitoring

In this session, participants were introduced to the objectives of the intervention, treatment rationale, and concept of experiential avoidance behaviors, which included unpleasant emotions, negative feelings, thoughts, stress, cravings, and urges.⁴⁹ Using texts, pictures, videos, and metaphors, participants would learn about the negative impact of avoidance behaviors related to alcohol consumption on various life domains, such as family, health, work, social life, and community. For instance, participants practiced the “dropping the struggle” metaphor and engaged in the “pushing away paper” exercise to understand how being hooked with negative emotions and cravings could affect them, and unhooking led them towards their life goals.

Participants were also encouraged to explore their past negative events, coping strategies, and the outcomes of these strategies, whether they provided long-lasting benefits or only temporary relief. Concurrently, participants were asked to identify and monitor their daily activities. They were required to list alcohol-free activities they currently and/or previously engaged in (e.g., taking a walk, spending time with family, playing football) and to evaluate their feelings about these activities using a daily activity monitoring form. This process helped them identify the most important and enjoyable activities.

Participants rated their activities based on enjoyment and importance on a scale from one to ten. This exercise helped identify low-rated activities as opportunities to replace them with more rewarding, value-driven alternatives. By engaging in these rewarding and alcohol-free activities, participants might better manage negative emotions and urges.

Session 2: Values and activity scheduling

In the second session, participants learned about their own values and life areas and how to schedule activities that aligned with their values. They assessed whether their current activities reflected the core values that could lead them to a more meaningful life. Short-term behavioral treatment goals, linked to the participant’s values, were developed to guide future behavioral activation efforts. Participants were asked to identify new value-driven activities to incorporate into their daily schedules across various life domains. They identified at least one value in each chosen life area. Participants were introduced to “value-driven activities,” defined as behaviors that aligned with their values, and they scheduled these newly identified activities. They used the value identification inquiry sheet and the Bull’s Eye tool to identify at least one value and schedule value-congruent activities.

Session 3: Acceptance and cognitive defusion

While practicing value-congruent activities, participants might encounter difficulties due to negative thoughts, emotions, cravings, and urges. In this session, participants learned acceptance and cognitive defusion skills. The concepts of acceptance and cognitive defusion, along with various experiential exercises, paradoxes, and metaphors, were explored. Once participants identified negative emotions, urges, and thoughts that led them to drink and affect their daily activities, they practiced acceptance and defusion skills and mindfulness exercises. These skills helped them accept negative thoughts and defuse negative emotions, urges, and cravings. They learned acceptance techniques by (1) allowing feelings, urges, and thoughts to occur without acting on them, (2) recognizing their strengths while acknowledging shortcomings, and (3) facing difficulties without avoidance. They also learned defusion skills by understanding that thoughts were transient and that impulsive thoughts about drinking were not necessarily facts.

Session 4: Being present and observing self

In addition to acceptance and defusion skills, participants learned mindfulness skills to be aware of the present moment and what was happening inside and outside their bodies. They engaged in exercises and metaphors for managing negative emotions, thoughts, cravings, urges, and memories related to past or future events that contributed to alcohol addiction. Participants also learned self-observation skills, understanding that there were parts of them that could think, feel, notice, and observe their urges, cravings, thoughts, and feelings toward alcohol use. They accessed mindfulness exercises to notice private events and used the ‘Sky and Weather’ metaphor for self-observation, introduced through video, audio, and written scripts.

Session 5: Daily plans and committed action

In this session, participants learned how to plan activities and take committed actions toward their life goals. They identified new relevant values and activities and integrated them into their daily lives. They learned to develop SMART goals and created their own life goals. Through practice, participants developed planning skills to schedule specific activities aligned with their daily values. They were encouraged to plan alcohol-free activities. In their daily plans, they scheduled value-driven activities. At the end of each day, they recorded the number of planned activities completed and rated the importance and enjoyment of each activity. The main goal of this practice was to establish regular engagement with rewarding activities, helping them identify new positive experiences and maintain involvement after treatment.

Session 6: Relapse prevention and planning

The final session focused on the post-treatment plan. Participants learned about the recovery process and relapse prevention, with a particular focus on preventing relapse and post-intervention planning. Previous sessions were reviewed, and another long-term goal congruent with their values was developed. Finally, participants were asked to develop a clear post-treatment plan based on their preferences and clinical significance.

Phase 2: Expert panel review

A local expert panel including two associated professors with numerous experiences in research in child and adolescent health and delivery of different psychosocial interventions, particularly ACT, an assistant professor with experiences in research of addiction and substance abuse, and three social workers in an NGO providing services for ethnic minorities in Hong Kong reviewed the appropriateness of the intervention, in relation to six items, i.e., (1) its relevance to the study objective; (2) relevancy of content; (3) accuracy; (4) meaningfulness; (5) comprehensiveness; and (6) ease of understanding by ethnic minority young adults. Each item was rated on a 4-point Likert-type scale ranging from 1 (irrelevant/inaccurate/incomprehensive/not meaningful/difficult to understand) to 4 (relevant/accurate/comprehensive/meaningful/easy to understand).

Phase 3: Development of an online platform for intervention delivery

Our intervention was delivered using Qualtrics, a secure and versatile platform designed for web-based survey development. This platform provides a cost-effective and efficient means for self-help interventions, significantly minimizing the need for extensive technical expertise, high development costs, and the time typically associated with creating web-based programs. Although Qualtrics is primarily designed for survey administration, it offers a range of essential features that are often expensive to implement on secure and stable websites. These features include various interactive question formats, database integration for tracking user behavior, the ability to customize information and carry data forward within a session, and reliable performance across multiple browsers and devices, including mobile platforms. If this approach is acceptable to participants, it presents a feasible option for researchers to deliver interventions using an online format at minimal or no cost.

The Qualtrics interface was carefully designed to facilitate easy access to information, interventions, and questionnaires for participants. A key aspect of this design was its shallow hierarchy and use of explicit labels, which enhanced navigability.

Phase 4: Usability test

The System Usability Test (SUT) was conducted before implementing the intervention with the target audience. The SUT is a method used to evaluate how effectively and efficiently users can interact with the internet-based interventions. It focuses on understanding user experiences, identifying usability issues, and collecting feedback to improve the overall design and functionality of the system. The objective of this usability testing was to assess the system’s ease of use, identify any usability problems that might affect user interaction, and evaluate user satisfaction.

Eligibility criteria

Participants for this usability testing were Hong Kong ethnic minority young adults aged 18 to 35 who can read, write, and speak English. Ethnic minorities in this age range were chosen as it aligns with the US Census Bureau classification for young adults [84], and this age group is highly vulnerable to substance use disorders [85]. According to the Hong Kong Department of Census and Statistics, ethnic minorities refer to persons who reported themselves as being of non-Chinese ethnicities in the population census.⁵⁰ They had to voluntarily participate, have internet-accessible devices, and meet the DSM-5 criteria for AUD.⁵¹ According to the DSM-5, participants exhibiting at least two of the following symptoms were considered to have AUD.⁵² The symptoms included: (1) Using alcohol in larger amounts or over a longer period than you intended; (2) Wanting to cut back on alcohol or making unsuccessful attempts; (3) Spending a great deal of time finding, using, or recovering; (4) Strong urges or cravings to use alcohol; (5) Use of alcohol that resulted in your inability to meet your obligations at work, home, or school; (6) Cut back on or abandon social, professional, or recreational activities due to your use of alcohol; (7) Repeatedly using alcohol when it was hazardous to do so; (8) Social or relationship problems due to your use of alcohol and kept using anyway; (9) Kept using alcohol knowing that it has caused physical or mental health issues; (10) Experienced uncomfortable physical or mental health symptoms (withdrawal); and (11) Diminished effects when you used alcohol compared to the past.

Sample size determination

Based on previous literature, at least five participants can provide adequate data to test the system usability.⁵³ Therefore, we recruited ten ethnic minority young adults with AUD for the system usability test.

Study setting, recruitment, and data collection

Using a convenience sampling technique, the study participants were recruited from bars in Chung King Mansion and Tsim Sha Tsui. These locations were chosen because it was easy to find target groups, as many ethnic minority young adults often gather there. Participants were provided with comprehensive information about the study. Their contact details, such as email, WhatsApp, or telephone numbers, were collected to receive an electronic invitation, which included a questionnaire to collect sociodemographic information, and the SUT with open-ended questions. Participants received a link to access a website with their usernames and passwords. If there was no response within one week after the invitation, reminders would be sent via email or SMS. Upon accessing the website, participants would be able to engage with the information, intervention modules, and questionnaires. They will be instructed to go through the intervention modules sequentially. Then, participants were asked to complete the SUT and provide comments for the open-ended questions. Recruitment and data collection were done from October 18 to December 31, 2024.

Measurement

A self-administered questionnaire was delivered to all participants to evaluate each module in terms of navigation, content clarity, instructions, ease of use, and module presentation. Participants responded to questions and could provide suggestions for the modules (Supplementary file 2).

System usability scale

The System Usability Scale (SUS) is a widely utilized 10-item measure designed to evaluate website usability, including aspects such as ease of navigation and user satisfaction.⁵⁴ This scale has a proven psychometric measurement, including validity and reliability.⁵⁵ The internal consistency of items in the previous study showed a Cronbach’s alpha of 0.86 ⁵⁵. Participants responded using a 5-point Likert scale, ranging from “strongly disagree” to “strongly agree.” The SUS score was calculated using the formula below (EQ 1),⁵⁶ where Un refers to the rating of the n^th question (Supplementary file 2). The SUS scores ranged from 0 to 100, with higher values suggesting higher user satisfaction. This score was compared with a cut-off of 70, a previously established metric.⁵⁶ The scores were then rated as follows: scores <25 = worst imaginable; 25 to 39.17 = poor; 39 to 52.5 =median; 53 to 73.5 =good; 74 to 85.5 =excellent, and 86 to 100 = best imaginable/possible.⁵⁶

SUS = 2.5 \times [\sum_{n = 1}^{5} (U_{2 n ‐ 1} ‐ 1) + (5 ‐ U_{n})]

Where:

SUS: System usability score

Un: the rating of the n^th question

n = question number

Equation 1: The formula used to calculate the overall score of the usability testing.

Data analysis

The authors checked, cleaned, and verified the data. Then, they imported the data into SPSS version 26 for analysis. Descriptive statistics were used to report the demographic characteristics of the participants, with mean and standard deviation for continuous variables and frequency and proportion for categorical variables, and median and interquartile range for non-normally distributed data. A one-sample t-test was conducted to determine whether the mean SUS score differed from the established metrics.

Ethical consideration

Ethical approval was obtained from the Human Subjects Ethics Sub-Committee of Hong Kong Polytechnic University (reference number: HSEARS20240712003). The protocol was registered on ClinicalTrials.gov under the National Clinical Trial number NCT06779006. Informed written consent was taken from each participant. They were informed about their right to decline or leave the study at any time. All data obtained from each participant was kept confidential in a non-identifying way.

Result

Phase 1: Preparation of self-help iACT-BA intervention content

A total of six sessions, with one module per week, were delivered to ethnic minority young adults with AUD. The modules included: (1) introduction, confronting agenda, and daily monitoring; (2) values and activity scheduling; (3) acceptance and diffusion; (4) being present and observing self; (5) daily plans and committed action; (6) relapse prevention and plan.

Phase 2: Expert panel review

The overall CVI was above the acceptable threshold (i.e., 0.8). However, two experts provided feedback for modifications, including avoiding the use of technical language, jargon, and difficult terms throughout the protocol, and incorporating withdrawal symptom management in the first session. Additionally, they recommended that audio or video-based experiential exercises should be used to make the content easier for participants to understand. Finally, a revision was made based on the expert’s recommendation, and then, the revised version was sent back to the panel if any further changes were needed.

Phase 3: Development of an online platform for intervention delivery

The Qualtrics interface included six modules, which presented sequentially. This sequential approach ensured that users fully comprehended one segment of information before advancing to the next. Pages were presented individually in each session. Moreover, the interface employed large fonts, simple text or video, and prominent buttons to improve users’ ability to view, understand, and interact with the content. To minimize distractions, the design intentionally excluded advertisements and moving content.

Phase 4: Usability testing

Sociodemographic characteristics

A total of 10 participants were invited to participate in usability testing of an internet-based self-help ACT-BA intervention. Of which, 9 completed the test, resulting in a completion rate of 90%. Participants predominantly consisted of Filipinos (77.8%) and Catholics (66.7%). The mean age of participants was 32.67 years (SD = 2.00). The majority of them were single (66.7%). Over two-thirds lived with nonrelatives (77.8%). About 66.7% held a college degree or higher, and most of them were working as domestic helpers (77.8%). The duration of residency in Hong Kong ranged from 1 to 19 years, with a median of 5 years (IQR: 1.5-7) (Table 1).

Table 1.

Sociodemographic characteristics of Ethnic minority young adults with AUD (n=9).

Variables	Participants (n = 9)	Proportion (%)
Age
Years M(SD)	32.67 (2.00)
Gender
Female	7	77.8
Male	2	22.2
Ethnicity
Filipino	7	77.8
Ugandan	2	22.2
Religion
Catholic	6	66.7
Orthodox Christians	2	22.2
Others	1	11.1
Marital status
Single	6	66.7
Married	1	11.1
Separated	2	22.2
Educational status
≤ 12 grades	3	33.3
College and above	6	66.7
Occupation
Domestic helper	7	78.8
Daily laborer	2	22.2
Living arrangements
Living alone	1	11.1
Living with parents/relatives	3	33.3
Living with non-relatives	4	44.4
Living with partners	1	11.1
Duration of living in Hong Kong
Median (IQR)	5 (1.5-7)

Note. M: Mean; SD: Standard deviation; IQR: Interquartile range.

System usability scale (SUS) score

As shown in Table 2, the SUS scores were computed among ethnic minority young adults with AUD. The Shapiro-Wilk test confirmed the normality of the SUS scores (p = 0.106). The overall usability score was 74.72 (SD=4.75), with the individual scores ranging from 70.5 to 85. According to the established benchmarks from the existing literature, the intervention had excellent usability.⁵⁶ A one-sample t-test comparing the mean SUS score to the established threshold of 70 revealed that our mean SUS score was notably higher (M = 74.72; SD = 4.75; p < 0.001). Besides, the scale demonstrated a good internal consistency (Cronbach’s α = 0.875).

Table 2.

Results of system usability scale (SUS).

Number	SUS questions	ACT-BA (n=9)
Number	SUS questions	M	SD
1	I think that I would like to use this system frequently.	4.11	.601
2	I found the system unnecessarily complex.	2.00	.50
3	I thought the system was easy to use.	4.00	.866
4	I think that I would need the support of a technical person to be able to use this system.	1.89	.601
5	I found the various functions in this system were well integrated.	4.00	.50
6	I thought there was too much inconsistency in this system.	2.22	.667
7	I would imagine that most people would learn to use this system very quickly.	4.11	.782
8	I found the system very cumbersome to use.	2.33	.50
9	I felt very confident using the system.	4.11	.782
10	I needed to learn a lot of things before I could get going with this system.	2.00	.707
	Total SUS Score	74.72	4.75
	T score	46.186
	P. Value	<0.001

Module evaluations

A total of nine participants completed a self-administered questionnaire that collected their feedback on the internet-based self-help intervention. The majority of them (88.9%) reported feeling great about the intervention, stating that it was useful to assist them in quitting alcohol. Additionally, seven participants noted that the flow and navigation of the content were effortless, and the information provided within the modules and instructions was adequate for them to complete the online intervention independently without assistance. However, three participants experienced technical difficulties, such as device incompatibility, internet interruptions, and login failures.

Overall, participants expressed high satisfaction with the intervention, with 88.9% (n = 8/9) finding it helpful, easy to navigate, and having clear content and instructions. Participants spent about 10–30 minutes per module, with 88.9% (n = 8/9) indicating that the website increased their confidence in completing modules. Almost all participants received at least one reminder per module to encourage completion. Seven participants completed all modules, while 2 completed five modules; however, one participant consented but did not respond to all modules (Table 3).

Table 3.

Module completion evaluation among ethnic minorities with AUD.

Intervention	N	%
Module completion
Completed all modules	7	77.7
Completed ≤ 5 modules	2	22.3
Technical errors
Yes	3	0.33
No	6	66.7
Usability testing completion	9	100
Self-administered question completion	9	100

Most participants (77.8%, n = 7/9) reported learning a lot from the modules, and 88.9% (n = 8/9) stated that it would change their perspective on drinking. While most participants stated that the flow and instructions were very clear, and technical support was not further needed, one participant faced a challenge in navigating the intervention due to problems accessing the internet through their devices. The four-phase intervention development process is summarized in Table 4.

Table 4.

The overall summary of the phases of intervention and protocol development.

Phases	Methods	Tools/sources	Results
1	Preparation of Self-help iACT-BA Intervention Content	ACT manual, self-help protocols, and behavioral activation treatment	A total of six modules, with one module per week, were developed. The modules include: (1) Introduction, confronting agenda, and daily monitoring; (2) Values and Activity scheduling; (3) Acceptance and cognitive defusion; (4) Being present and observing self; (5) Daily plans and committed action; (6) Relapse prevention and plan.
2	Expert panel review	Content validity index	The CVI was acceptable (i.e., 0.8). However, two experts provided feedback for modifications, including avoiding the use of technical language, jargon, and difficult terms, and incorporating withdrawal symptom management. Additionally, recommended to add audio or video-based experiential exercises. Finally, a revision was made based on the expert’s recommendation, and then the revised version was sent back to the panel if any further changes were needed.
3	Development of an online platform for intervention delivery	Qualtrics	The Qualtrics interface included six modules, which presented sequentially. Pages were presented individually in each session. The interface employed large fonts, simple text or video, and prominent buttons to improve users’ ability to view, understand, and interact with the content.
4	Usability testing	System usability scale and self-administered questionnaire	The overall usability score was 74.72 (SD=4.75), with the individual scores ranging from 70.5 to 85. A one-sample t-test comparing the mean SUS score to the established threshold of 70 revealed that our mean SUS score was notably higher (M = 74.72; SD = 4.75; p < 0.001).
4	Usability testing	System usability scale and self-administered questionnaire	Regarding module evaluations, the majority (88.9%) reported feeling great. Seven participants noted that the flow and navigation of the content were effortless, and the information provided within the modules and instructions was adequate for them to complete the intervention online independently without assistance. However, three participants experienced technical difficulties, such as device incompatibility, internet interruptions, and login failures.

Discussion

A significant number of ethnic minorities are affected by AUD. A few of the major reasons were acculturation stress, avoidance behaviour, stigma, and discrimination. To address these reasons, ACT-BA was found as a viable option. Before the pilot and full-scale RCT to examine the feasibility and effectiveness, usability testing is a key component for the intervention development. Therefore, this proof-of-concept aimed to develop and test the usability of a web-based self-help ACT-BA intervention for ethnic minority young adults with AUD in Hong Kong. The findings provide us with valuable information to identify areas of improvement to finalize the intervention.

In phase 1, two authors have developed a protocol from different sources, including the ACT manual,^47,48 self-help protocols,²⁸ and behavioral activation treatment.^37,38 The content was adapted from this evidence and prepared into six self-help modules for ethnic minority young adults. The modules included introduction, confronting agenda, and daily monitoring; values and activity scheduling; acceptance and cognitive defusion; being present and observing self; daily plans and committed action; and relapse prevention and planning. The content of each module was developed according to its purpose and relations; for instance, we have prepared acceptance and cognitive defusion in one module, with acceptance content coming first, then cognitive defusion. The sequence and number of sessions were determined based on the existing literature.^57,58 Consistent with previous studies,^28,57 study participants were allowed one week per module, for a total of six weeks for six modules.

In phase 2, a local expert panel comprised of professionals with expertise in addiction and ACT implementation was rigorously reviewed and suggested modifications to the intervention before conducting the usability testing. Their evaluations focused on the intervention’s content, accuracy, comprehensiveness, meaningfulness, relevance to the study’s objective, and ease of understanding for ethnic minority young adults. Most of the items were considered quite or highly relevant by most experts, which made the content validity index acceptable, aligned with its intended purpose and population. Despite this positive evaluation, the experts provided insightful recommendations for refinement in future iterations. Firstly, the panel experts suggested simplifying the technical ACT language. This comment was particularly essential for ethnic minorities with lower educational levels, who may struggle with complex terms. The panel members specifically recommended that the technical terms of ACT should be translated into plain and simple language, thus enabling the ethnic minorities to self-complete the intervention modules without assistance. Additionally, experts suggested that audio and video-based metaphors and exercises should be used instead of only providing textual descriptions. This provided additional benefits, particularly for visual learners and individuals with limited proficiency in language comprehension. Another crucial recommendation was the incorporation of withdrawal symptoms management in the first session of intervention. One expert stated that participants might experience withdrawal symptoms immediately after deciding to quit alcohol. Focusing on these symptoms in the first module could enhance engagement and adherence to the subsequent sessions. Following a rigorous refinement based on the expert feedback, in phase 3, an online intervention delivery platform was developed for ethnic minority young adults. This modality offers several advantages over the traditional face-to-face format.^41–43 Firstly, online intervention is highly accessible for participants living in remote areas where access to healthcare services is limited.⁵⁹ Additionally, findings from our previous qualitative and quantitative studies have indicated that most ethnic minority young adults in Hong Kong are employed and have limited free time.¹⁸ This makes it difficult to attend in-person sessions or participate in various supporting groups or organizations. All these shortcomings of in-person modalities can be solved through online interventions.⁵⁹ Furthermore, existing evidence supports the feasibility and effectiveness of online interventions for AUD.⁶⁰ In this regard, WHO has recommended the adoption of digital health interventions to increase healthcare coverage, particularly for diverse and hard-to-reach populations.⁶¹ This growing emphasis on digital health and artificial intelligence-assisted healthcare services provides a broader and more efficient dissemination of interventions.^61,62

Apart from the panel’s suggestion, in phase 4, a usability test was conducted in a group of ethnic minority young adults. The results of our one-sample t-test showed that the mean SUS score among the participants markedly higher than the threshold, suggesting that the intervention content was appropriate to the target population. Also, the mean score of three items, including (1) I think that I would like to use this system frequently; (2) I would imagine that most people would learn to use this system very quickly; and (3) I felt very confident using the system, was higher than 4, reflecting that our intervention could be completed independently by the participants without any assistance.

Conducting this proof of concept was instrumental in revealing areas that needed further improvement. The process evaluations showed both strengths and challenges with this platform. Most participants felt happy with the intervention and were actively engaged in the intervention. Participants completed most modules, including weekly assignments and exercises. Most participants stated that the module was clear, coherent, understandable, and helpful, despite a few technical errors and some difficulties in understanding a few words, which needed self-searching. Future interventions could optimize the content by further simplifying the language. Additionally, a few participants experienced difficulties in navigating the content and reported technical challenges, including device incompatibility, internet interruptions, login failures, and other technical errors. These problems need to be solved for future interventions, for instance, by considering a brief session (∼10 minutes) to go through the intervention together with the participants using their mobile phones or electronic devices during subject recruitment and ensuring the intervention to be functional on multiple electronic devices and in different operating systems, e.g., Android and iOS. Besides the technical challenges, most participants needed frequent reminders for module completion. Notably, almost all participants needed at least one reminder per module. However, the timing of reminders is incredibly important for ethnic minority young adults. Many participants reported being quite busy during weekdays and unable to use mobile devices for personal activities, often due to work restrictions by their employers. They indicated that they were mostly available on Sundays, holidays, or late at night, and sending reminders during these periods could substantially enhance engagement and module completion for future intervention in this segment of the population.

Limitations

Despite its novelty, this study has some limitations. First, since the authors used online self-administered questionnaires, they did not employ a think-aloud technique to capture their thoughts. Second, the self-reported outcome measurement may have led to underestimating or overestimating the outcome variable. Third, the sample was small, which limited the statistical power of SUS scores and the generalizability of findings to the broader population.

Future interventions

Intervention protocol development and usability testing are key preliminary steps before actual implementation. The results of usability testing indicated that the internet-based self-help ACT-BA intervention was appropriate for ethnic minority young adults. However, a few modifications would be made based on the suggestions provided by experts and end-users. Future research will conduct a brief session to guide the participants to access the intervention using their smart devices during subject recruitment, ensure that the intervention content can be accessed using different smart devices and operating systems, and frequently send reminders to participants, especially during holidays and weekends, to improve their engagement. Interventions could also be prepared in plain and simple language and can incorporate withdrawal symptom management in the first session.

Conclusion

A new internet-based self-help ACT-BA intervention was developed and evaluated for its usability among Hong Kong ethnic minority young adults with AUD. The findings of this usability testing showed an excellent usability score. However, minor technical errors and layout issues were identified. Future studies should consider minimal modifications. The efficacy and feasibility of our proposed intervention shall be further examined in the future randomized controlled trial.

Supplemental material

Supplemental material - Development and usability testing of an internet-based self-help program integrating acceptance and commitment therapy and behavioral activation intervention for ethnic minority young adults with alcohol use disorder: Usability study

Supplemental material for Development and usability testing of an internet-based self-help program integrating acceptance and commitment therapy and behavioral activation intervention for ethnic minority young adults with alcohol use disorder: Usability study by Getaneh Mulualem Belay, Katherine Ka Wai Lam, Qi Liu, Ting Mao, Qian Sun, Yim Wah Mak, Ka Yan Ho in DIGITAL HEALTH.

Supplemental material

Supplemental material - Development and usability testing of an internet-based self-help program integrating acceptance and commitment therapy and behavioral activation intervention for ethnic minority young adults with alcohol use disorder: Usability study

Footnotes

Acknowledgement

Firstly, we would like to express our gratitude to the Hong Kong Polytechnic University for granting the Qualtrics platform to design this internet-based self-help intervention. Secondly, we want to express our sincere acknowledgment to all study participants who took part in this study. Finally, the authors would like to thank the expert review panel for their expertise in evaluating the intervention protocol.

ORCID iD

Ka Yan Ho

Ethical considerations

This study received ethical approval from the Human Subjects Ethics Sub-Committee of the Hong Kong Polytechnic University (reference number: HSEARS20240712003). The protocol was registered on ClinicalTrials.gov with a National Clinical Trial number of NCT06779006. Informed written consent was taken from each participant.

Author Contributions

GMB: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization, Writing, review, and editing; KKWL: Methodology, Validation, Visualization, review, and editing, QL: Methodology, Visualization, review, and editing, TM and QS: Methodology, review, and editing, YWM: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Visualization, Supervision, Writing, review, and editing, and KYH: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Validation, Supervision Visualization, Writing, review, and editing

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Declaration of conflicting interests

The authors declare no conflict of interest.

Data Availability Statement

The datasets generated or analyzed during this study are available from the corresponding author on reasonable request.*

Trial registration

ClinicalTrials.gov: /NCT06779006.

Guarantor

KYH.

Supplemental material

Supplemental material for this article is available online.

References

Connor

Haber

Hall

. Alcohol use disorders. The Lancet 2016; 387: 988–998. https://doi.org/10.1016/S0140-6736(15)00122-1

Delker

Brown

Hasin

. Alcohol consumption in demographic subpopulations: an epidemiologic overview. Alcohol research: current reviews 2016; 38: 7–15.

Erol

Karpyak

. Sex and gender-related differences in alcohol use and its consequences: Contemporary knowledge and future research considerations. Drug and alcohol dependence 2015; 156: 1–13. https://doi.org/10.1016/j.drugalcdep.2015.08.023

Caetano

Gruenewald

Vaeth

, et al. DSM‐5 Alcohol Use Disorder Severity in Puerto Rico: Prevalence, Criteria Profile, and Correlates. Alcoholism: clinical and experimental research 2018; 42: 378–386. https://doi.org/10.1111/acer.13572

Grant

Chou

Saha

, et al. Prevalence of 12-month alcohol use, high-risk drinking, and DSM-IV alcohol use disorder in the United States, 2001-2002 to 2012-2013: results from the National Epidemiologic Survey on Alcohol and Related Conditions. JAMA psychiatry 2017; 74: 911–923. https://doi.org/10.1001/jamapsychiatry.2017.2161

Belay

Wai

KLK

Mak

, et al. Factors associated with alcohol use disorder among young adults from ethnic minorities: a cross-sectional study. BMC psychology 2026.

Spadola

Wagner

Accornero

, et al. Alcohol use patterns and alcohol use disorders among young adult, ethnically diverse bariatric surgery patients. Substance abuse 2017; 38: 82–87. https://doi.org/10.1080/08897077.2016.1262305

Grant

Vergés

Jackson

, et al. Age and ethnic differences in the onset, persistence and recurrence of alcohol use disorder. Addiction 2012; 107: 756–765. https://doi.org/10.1111/j.1360-0443.2011.03721.x

Buckner

Lewis

Shepherd

, et al. Ethnic discrimination and alcohol-related problem severity among Hispanic/Latin drinkers: The role of social anxiety in the minority stress model. Journal of substance abuse treatment 2022; 138: 108730. https://doi.org/10.1016/j.jsat.2022.108730

10.

Dominic

Michelle

. Being Young and a Minority: A Study of the Challenges encountered by Young Ethnic Minorities in Hong Kong. Journal of Youth Studies (10297847) 2011; 14. https://shorturl.at/jeR1i

11.

Glass

Williams

. Racial/ethnic discrimination and alcohol use disorder severity among United States adults. Drug and alcohol dependence 2020; 216: 108203. https://doi.org/10.1016/j.drugalcdep.2020.108203

12.

Torres

Williams

Ceballos

, et al. Discrimination, acculturative stress, alcohol use and their associations with alcohol-related consequences among Latino immigrant men. Journal of Ethnicity in Substance Abuse 2024; 23: 150–165. https://doi.org/10.1080/15332640.2022.2077273

13.

L-T

Pilowsky

Schlenger

, et al. Alcohol use disorders and the use of treatment services among college-age young adults. Psychiatric Services 2007; 58: 192–200. https://doi.org/10.1176/ps.2007.58.2.192

14.

Belay

Mak

Wong

FKY

, et al. Psychosocial treatment options for adolescents and young adults with alcohol use disorder: systematic review and meta-analysis. Frontiers in public health 2024; 12: 1371497. https://doi.org/10.3389/fpubh.2024.1371497

15.

Belay

Lam

KKW

Liu

, et al. Magnitude and determinants of alcohol use disorder among adult population in East Asian countries: A systematic review and meta-analysis. Frontiers in Public Health 2023; 11: 1144012. https://doi.org/10.3389/fpubh.2023.1144012

16.

Salom

Williams

Najman

, et al. Familial factors associated with development of alcohol and mental health comorbidity. Addiction 2015; 110: 248–257. https://doi.org/10.1111/add.12722

17.

Lieb

Merikangas

Höfler

, et al. Parental alcohol use disorders and alcohol use and disorders in offspring: a community study. Psychological medicine 2002; 32: 63–78. https://doi.org/10.1017/s0033291701004883

18.

Belay

Wah

Lam

KKW

, et al. Reasons and impacts of alcohol use disorder among ethnic minority young adults: a descriptive phenomenological study. Frontiers in Public Health 2025; 13: 1560092. https://doi.org/10.3389/fpubh.2025.1560092

19.

Hayes

Strosahl

. A practical guide to acceptance and commitment therapy. Springer Science & Business Media, 2004.

20.

Sevinçok

Memiş

ÇÖ

Çakaloz

. Acceptance and commitment therapy experience in an adolescent patient with social anxiety disorder, panic disorder, and major depression. Psychiatry and Clinical Psychopharmacology 2018; 28: 138–139.

21.

Hayes

. Acceptance and commitment therapy, relational frame theory, and the third wave of behavioral and cognitive therapies. Behavior therapy 2004; 35: 639–665. https://doi.org/10.1016/s0005-7894(04)80013-3

22.

George

de Guzman

. Effectiveness of acceptance and commitment therapy based intervention program (ACTP) on perceived stress and emotion regulation among alcoholics in Kerala, India. Indian Journal of Positive Psychology 2015; 6.

23.

Thekiso

Murphy

Milnes

, et al. Acceptance and commitment therapy in the treatment of alcohol use disorder and comorbid affective disorder: a pilot matched control trial. Behavior Therapy 2015; 46: 717–728. https://doi.org/10.1016/j.beth.2015.05.005

24.

Petersen

Zettle

. Treating inpatients with comorbid depression and alcohol use disorders: A comparison of acceptance and commitment therapy versus treatment as usual. The Psychological Record 2009; 59: 521–536. https://doi.org/10.1007/bf03395679

25.

Weststrate

Briggs

Miller

, et al. Brief acceptance and commitment therapy added to medication management during acute alcohol detoxification: A pilot randomized controlled effectiveness trial. Journal of Contextual Behavioral Science 2023; 29: 76–85. https://doi.org/10.1016/j.jcbs.2023.06.002

26.

Khandelwal

Das

Sharma

, et al. Testing the waters: A pilot trial of acceptance and commitment therapy (ACT) for alcohol use disorder. Indian Journal of Psychiatry 2024; 66: 191–194. https://doi.org/10.4103/indianjpsychiatry.indianjpsychiatry_820_23

27.

Meyer

Walser

Hermann

, et al. Acceptance and commitment therapy for co‐occurring posttraumatic stress disorder and alcohol use disorders in veterans: Pilot treatment outcomes. Journal of Traumatic Stress 2018; 31: 781–789. https://doi.org/10.1002/jts.22322

28.

Levin

Haeger

Pierce

, et al. Web-based acceptance and commitment therapy for mental health problems in college students: A randomized controlled trial. Behavior modification 2017; 41: 141–162. https://doi.org/10.1177/0145445516659645

29.

Brown

Hooper

James

, et al. A web-delivered acceptance and commitment therapy intervention with email reminders to enhance subjective well-being and encourage engagement with lifestyle behavior change in health care staff: randomized cluster feasibility stud. JMIR formative research 2020; 4: e18586. https://doi.org/10.2196/18586

30.

Clark

. Learning theories: behaviorism. Am Soc Radiol Tech 2018: 172–175.

31.

Martell

Dimidjian

Herman-Dunn

. Behavioral activation for depression: A clinician's guide. Guilford Publications, 2021.

32.

Miguel

Yamauchi

Simões

, et al. From theory to treatment: Understanding addiction from an operant behavioral perspective. Journal of Modern Education Review 2015; 5: 778–787. https://doi.org/10.15341/jmer(2155-7993)/08.05.2015/006

33.

Banyan . Operant Conditioning & Addiction, 2022.

34.

Mazzucchelli

Kane

Rees

. Behavioral activation treatments for depression in adults: a meta‐analysis and review. Clinical Psychology: science and practice 2009; 16: 383–411. https://doi.org/10.1111/j.1468-2850.2009.01178.x

35.

Dimidjian

Barrera

Martell

, et al. The origins and current status of behavioral activation treatments for depression. Annual review of clinical psychology 2011; 7: 1–38. https://doi.org/10.1146/annurev-clinpsy-032210-104535

36.

Fernández-Rodríguez

Coto-Lesmes

Martínez-Loredo

, et al. Is activation the active ingredient of transdiagnostic therapies? A randomized clinical trial of behavioral activation, acceptance and commitment therapy, and transdiagnostic cognitive-behavioral therapy for emotional disorders. Behavior modification 2023; 47: 3–45. https://doi.org/10.1177/01454455221083309

37.

Gaudiano

Nowlan

Brown

, et al. An open trial of a new acceptance-based behavioral treatment for major depression with psychotic features. Behavior modification 2013; 37: 324–355. https://doi.org/10.1177/0145445512465173

38.

Daughters

Magidson

Lejuez

, et al. LETS ACT: A behavioral activation treatment for substance use and depression. Advances in Dual Diagnosis 2016; 9. https://doi.org/10.1108/add-02-2016-0006

39.

Frohlich

Stewart

Korsapathy

, et al. Single-session behavioral activation for alcohol use disorder: a randomized controlled pilot trial. Cognitive Behaviour Therapy 2025; 55: 1–27. https://doi.org/10.1080/16506073.2025.2512146

40.

Carlbring

Hägglund

Luthström

, et al. Internet-based behavioral activation and acceptance-based treatment for depression: a randomized controlled trial. Journal of affective disorders 2013; 148: 331–337. https://doi.org/10.1016/j.jad.2012.12.020

41.

Moock

. Support from the internet for individuals with mental disorders: advantages and disadvantages of e-mental health service delivery. Frontiers in Public Health 2014; 2: 65. https://doi.org/10.3389/fpubh.2014.00065

42.

Taylor

Luce

. Computer-and internet-based psychotherapy interventions. Current directions in psychological science 2003; 12: 18–22. https://doi.org/10.1111/1467-8721.01214

43.

Andersson

Titov

. Advantages and limitations of Internet‐based interventions for common mental disorders. World Psychiatry 2014; 13: 4–11. https://doi.org/10.1002/wps.20083

44.

Molfenter

Roget

Chaple

, et al. Use of telehealth in substance use disorder services during and after COVID-19: online survey study. JMIR mental health 2021; 8: e25835. https://doi.org/10.2196/25835

45.

Ndulue

Naslund

. Digital interventions for alcohol use disorders: A narrative review of opportunities to advance prevention, treatment and recovery. Psychiatry Research Communications 2024; 4: 100183. https://doi.org/10.1016/j.psycom.2024.100183

46.

Balhara

Verma

. A review of web based interventions focusing on alcohol use. Annals of medical and health sciences research 2014; 4: 472–480. https://doi.org/10.4103/2141-9248.139272

47.

Hayes

. The six core processes of ACT. https://contextualscience.org/the_six_core_processes_of_act (2006).

48.

Hayes

. The six core processes of ACT: The psychological flexibility model.

49.

Hayes

Strosahl

. A practical guide to acceptance and commitment therapy. Springer Science & Business Media, 2004.

50.

Department CaS . 2021 Population Census – Thematic Report: Ethnic Minorities, 2022.

51.

National Institute . Alcohol Use Disorder: A Comparison Between DSM–IV and DSM–5 on on Alcohol abuse or Alcoholism. https://www.niaaa.nih.gov/publications/brochures-and-fact-sheets/alcohol-use-disorder-comparison-between-dsm (2021).

52.

National Institute on Alcohol Abuse and Alcoholism . Alcohol Use Disorder: A Comparison Between DSM–IV and DSM–5. https://www.niaaa.nih.gov/publications/brochures-and-fact-sheets/alcohol-use-disorder-comparison-between-dsm (2021).

53.

Henstam

. How many participants are needed when usability testing physical products? An analysis of data collected from usability tests conducted on physical products, 2018.

54.

Brooke

. SUS: a retrospective. Journal of usability studies 2013; 8.

55.

Khan

Hickie

Loblay

, et al. Psychometric evaluation of the System Usability Scale in the context of a childrearing app co-designed for low-and middle-income countries. Digital Health 2025; 11: 20552076251335413. https://doi.org/10.1177/20552076251335413

56.

Bangor

Kortum

Miller

. An empirical evaluation of the system usability scale. Intl Journal of Human–Computer Interaction 2008; 24: 574–594. https://doi.org/10.1080/10447310802205776

57.

Batink

Bakker

Vaessen

, et al. Acceptance and commitment therapy in daily life training: a feasibility study of an mHealth intervention. JMIR mHealth and uHealth 2016; 4: e103. https://doi.org/10.2196/mhealth.5437

58.

Hayes

Pierson

. Acceptance and commitment therapy. Encyclopedia of cognitive behavior therapy. Springer, 1999; 1–4.

59.

Banbury

Nancarrow

Dart

, et al. Telehealth interventions delivering home-based support group videoconferencing: systematic review. Journal of medical Internet research 2018; 20: e25. https://doi.org/10.2196/jmir.8090

60.

Gonzalez

Dulin

. Comparison of a smartphone app for alcohol use disorders with an Internet-based intervention plus bibliotherapy: A pilot study. Journal of consulting and clinical psychology 2015; 83: 335–345. https://doi.org/10.1037/a0038620

61.

World Health Organization . Digital health.

62.

Gou

Liu

Xiao

, et al. Research on artificial-intelligence-assisted medicine: a survey on medical artificial intelligence. Diagnostics 2024; 14: 1472. https://doi.org/10.3390/diagnostics14141472

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.17 MB

0.53 MB