Abstract
The Problem
When the global economy went into recession, many organizations needed to revisit their training investment. In this case study, one organization wanted to retain its emphasis on training without committing to the flight and hotel costs associated with in-person delivery for a globally distributed workforce. The training department therefore looked for a virtual training solution informed by research and theory that could host existing in-person training courses in a virtual environment while also providing a professional experience and high evaluation ratings.
The Solution
The organization selected a 3D virtual world to host a custom-built training environment consisting of an auditorium, a training center, a student center, and interactive displays. Within 5 months, the environment was hosting regular multiday training courses that delivered high evaluation ratings while saving the company just under US$1,000,000 over a 2-year period. The case study describes the environment as well as the lessons learned from the implementation.
The Stakeholders
This article provides scholar-practitioners with an example of how research and theory can inform practice to create a better solution for a real-life organizational problem. It also provides scholar-practitioners with guidance on how to explore 3D virtual worlds for use in their own organizations and in training delivery more widely.
According to Short and Shindell (2009), a human resource development (HRD) scholar-practitioner is distinguished from others working in HRD by four activities: (a) they ground their practice in research and theory; (b) they champion research and theory in the workplace; (c) they conduct and disseminate research, and partner with academic researchers; and (d) they act as a bridge between research and practice in seeking further development of the field of HRD. Like scholar-practitioners in other disciplines and fields, those in HRD face multiple challenges, including gaining access to research and finding research of direct relevance to their problem at hand (Short, Keefer, & Stone, 2009).
The case study in this article occurs in the intersection of HRD and technology, where problems are relatively new and so are poorly understood and ill-defined. It illustrates how a scholar-practitioner approached a problem on the leading edge of HRD practice, and used research and theory to inform the solution. In doing this, the case study shows how a scholarly approach can inform practice, and also shows how scholar-practitioners can find a path even when working on areas where there is currently little HRD research.
Problem
The case study takes place in an organization that was founded at the start of the century, and provides business technology solutions and managed services to customers around the world. For ease of reading, the company is referred to as ABC throughout the article. ABC employs over 17,000 people in more than 20 countries, and is headquartered in the United States. Most of its employees are technology experts. ABC has always placed a heavy emphasis on employee development, including requiring annual performance management goals on training hours, providing employees with chargeability relief for training, and offering widespread financial and time support for professional certification.
In 2009, the global economy was in recession, and companies around the world were feeling the impact of lower capital investments, lower customer and business confidence, and falling GDPs. Like the majority of companies around the world, ABC needed to control expenses and investments during a challenging period. The decision was made to retain the company’s traditional emphasis on employee development by exploring ways of moving in-person training into a virtual format. To achieve this, ABC needed highly effective virtual training for multiday courses on management, consulting skills, and project management; and for these courses to be available to any employee in 20+ countries. ABC’s training department therefore created a project to identify a virtual environment suitable for hosting training previously held in-person. The project team consisted of a senior training manager, representatives of the company’s training instructors, and an administrator/operations lead.
To meet stakeholder requirements, that environment needed to be accessible to all employees, deliver high levels of student satisfaction and learning effectiveness, be company-branded, and have a professional/corporate look-and-feel. The project team agreed with stakeholders that the new virtual campus would be operational for testing within three months, and would host training courses within 6 months. It was also agreed that ABC’s expenditure on the project would not exceed 10% of the cost to host the training in-person. The project team therefore faced a tight timeline to identify and create a virtual environment capable of hosting training previously run in-person, and to deliver those courses with minimal impact to course evaluations. The remainder of this case study describes the process used to identify a solution, the design of that solution, and the performance of that solution postrelease.
Process to Reach a Solution
Prior to this project, ABC had delivered training through virtual formats for 8 years. From 2001 to 2004, that took the form of 4- or 5-day virtual boot camps on technical topics taught via desktop-sharing tools that combined voice and text chat interactions between instructors and students. From 2006 to 2008, a similar process was followed for a major required onboarding program when new hires completed 12 hr of virtual classroom time over a 3-day period that were blended with 9 hr of e-learning. Although the tool used for these two courses was reliable, it was best suited to knowledge-based training with a focus on information-sharing and basic interactions (e.g., technology training), as opposed to the skills-based training (e.g., customer interaction skills) that was the focus of the company’s need by 2009. ABC therefore had a need to explore alternative virtual training delivery tools that were better placed to support skills development.
To identify a solution, the project team began a first phase focused on finding a tool that allowed for more student interaction than generally found in most virtual training. A second phase would then focus on creating an environment that interested students who had previously been used to attending in-person training, and that could retain student attention and learning for the same 8-hr days as in-person training. That second phase would continue into building and implementing the new approach within 3 months so that employee access to training would not be removed for a lengthy period.
As part of the first phase, those running the project within ABC examined practitioner articles as well as scholarly research and theory to assist with selecting a tool and implementing it. The information used came from three main sources: a literature review, interviews, and meetings with vendors. The literature review was completed through searches of ProQuest, Google, and Google Scholar using such key phrases as: virtual instruction; blended learning; and converting in-person training to virtual. The search phrases were selected based on the project team members’ past experiences of virtual training. Additional phrases were added in a second phase of the literature review that built on articles identified in the first phase. The very tight timeline for the project required that the project team complete the literature search within 1 week. Had this been a research study, a more thorough literature review would have been required; however, as this was an organizational problem demanding fast action, the project team needed to complete a rapid review of the literature and move forward.
It was quickly identified that standard 2D learning environments, as used in most webinars and similar events, would not meet the company’s needs for interactivity and learning. However, the project team identified examples of how emerging technologies in 3D virtual worlds were being used in education and training, and this became the focus on the team’s work.
The literature review focused on the limited literature available on learning in 3D environments, and was completed at the start of the project. This identified a number of potentially useful sources, including Bardzell and Odom (2008), Bartle (2003), Bowman and McMahan (2007), Chen, Toh, and Wan (2004), Chen, Toh, and Wan (2005), de Freitas and Neumann (2009), Hew and Cheung (2010), Jarmon, Traphagan, Mayrath, and Trivedi (2009), Kahai, Carroll, and Jestice (2007), and Mayer (2002). The literature review also identified case studies of how 3D learning environments had been implemented by the U.S. military as well as by private sector organizations such as Intel. The analysis of the literature review was used to: Identify organizations that had already used 3D virtual worlds for corporate training, and to explore case studies of such organizations; Identify critical issues in the building and implementation of the virtual campus; and Identify references and sources that could be used to validate and justify our overall direction when meeting with key stakeholders within the company.
Interviews included discussions with practitioners in other organizations, as well as with authors of published case studies and an author of a leading book on learning in 3D (Kapp & O’Driscoll, 2010). These meetings were used to discuss best practices, lessons learned, and options for ways forward; as well as for identifying potential vendors that would meet the company’s needs for a research-informed solutions.
The vendor meetings allowed ABC to meet with several vendors specializing in virtual platforms. These meetings reviewed examples of designs for other organizations; explored what works and doesn’t work in practice; examined how training in-person could be adapted to training in virtual worlds; and increased the understanding of the design and build phases of the process.
In addition to external information, the project team also obtained information from within ABC. This included seeking inputs from employees, key stakeholders and business leaders, and instructors. Those meetings continued throughout the project and into the implementation phase to ensure that solution met the needs of the different stakeholders. Throughout, options and solutions were assessed against criteria derived from the original project goals:
Could the solution deliver highly effective virtual training for multiday courses on management, consulting skills, and project management?
Could the solution deliver training to employees in 20+ countries?
Would the solution receive high levels of employee satisfaction from those attending training courses?
Could the solution be branded, and so be easily identified by employees as a formal part of the ABC training department?
Could the environment be ready for operational testing within 3 months, and be ready to host training courses within 6 months?
Would ABC’s expenditure on the project not exceed 10% of the cost to host the training in-person?
Solution
After considering several options, the company opted for a custom-built virtual campus in Second LifeTM. Second LifeTM is an Internet-based 3D virtual world. Although, at the time, Second LifeTM was better known for use by individuals as a gaming or socialization tool rather than for enterprise solutions, it had an emerging track record of use in education and corporate training/conferences. Despite its lack of a long-term record, the solution was still considered a better option than any other tool on the market at the time. In the absence of an off-the-shelf solution within Second LifeTM, ABC worked with an external vendor to build a custom virtual campus that had the look and feel of a bricks-and-mortar corporate training campus. The virtual campus was designed to meet the project goals by ensuring high levels of student satisfaction and learning effectiveness, allowed for access to any ABC employee, included corporate branding, and provided a professional feel. The solution design was also informed by the literature review, meetings with authors, and meetings with vendors. Between them, these sources guided the project team in identifying the components, and the specific designs of each component. The components included:
A company-branded landing hub. This was where employees arrived when first visiting the campus. It contained easily accessible campus rules (approved by the company Legal and HR departments), a self-guided walking tour of the campus, a scale-model of the campus to assist new arrivals with orientation, and a reception desk with the flags of every country where ABC operates.
A training room. This was capable of seating 20 students in a traditional U-shape, with a PowerPoint slide projector and a white board, a podium for presenters, and additional items typically found in corporate training settings (e.g., a refreshments table with virtual drinks and sandwiches). During courses, students could listen and watch instructors and PowerPoints, discuss the course content in a local typed chat window, and discuss issues one-to-one with an instructor via personal typed chat or voice chat. The design allowed for considerable interaction between students, and between instructors and students.
Breakout rooms. Around the training room was a bank of five breakout rooms, each containing a table with 8 chairs, a white board, and an Internet browser. These breakout rooms were used for small-group activities in the same way that they would during in-person training. Each room was on a separate sound server so that discussions in those rooms were private to the students within them.
Meeting room. Adjacent to the training room was an executive meeting room consisting of a board room, executive chairs, a PowerPoint screen and projector, tea/coffee, and other items typically found in a board room. This room was generally used for small meetings or for customer-based role-plays where students would practice entering a customer meeting room to discuss a consulting project. During such role-plays, other students could observe and listen from the sidelines prior to feedback being offered by instructors.
Student center. The campus included a student center with a coffee bar (allowing students to network during breaks between training sessions and after courses), and with a library showing the 100 top e-books used by company employees (the library was designed like a standard library with books showing their covers, and students would click on a book and it would open as an e-book in a browser).
Interactive displays. The campus included a series of three interactive displays that students could explore during visits to the campus. These were designed to offer an alternative learning style, with an emphasis on employees interacting with the content rather than participating in a training course. They were therefore designed from a gaming perspective where employees learned via participation without the involvement of an instructor. One such interactive display was a maze based on the company’s orientation for new hires that contained material from the orientation and students would have to answer questions about the material to open gates that allowed them into the next section of the maze. A second interactive display was a “trade show” demonstrating the company’s products. For each product, there were slides describing the product and a quiz to check for understanding. A third was an interactive pathway describing training and learning available to employees to help them learn about access to training (clicking on images would open up additional information in a browser window).
To meet the tight project deliverable deadline, the design, development and implementation work was all completed in a short timeframe. Initial research on best practices and case studies was finished by February 28, 2010. The company project team then met with external subject matter experts, and proposed a solution to business stakeholders by March 31, followed by a decision to proceed made by senior stakeholders by April 1. Contracting with vendors on design and build phases started on April 12, with the conceptual design agreed upon and signed off by April 30. The virtual campus was built and ready for testing by June 15, and testing was completed by June 30. Early adopters entered into the campus by July 15, supported by orientation sessions and the first 1-hr presentations. The first 2-day training course was adapted for the virtual campus, advertised to employees, and running in the environment by July 31; and all three 2-day training courses were operating fully within the campus by August 31, supported by fully functioning registrations and orientation sessions.
In support of the release, there were a number of key activities aimed at preparing employees, stakeholders, and instructors. These included email and phone call briefing sessions with key stakeholders during the design and build stages, to ensure they were aware of the specifics and had an input. These key stakeholders were then allowed access to the environment before other employees to increase understanding and ownership. Once the environment was opened to employees, they were given orientation sessions by trained “greeters” prior to needing to attend the campus for a training course. This allowed for resolving any unexpected technical issues and increasing employees’ levels of comfort when operating in the environment. Employees were also provided with instructions on using the campus, posted on the ABC intranet site, including videos/demos of how to resolve common challenges and how to make the most of the experience. In addition, prior to offering multiday training courses, the team organized shorter (1-hr) presentations that allowed instructors to practice instructing in the environment, allowing students to practice learning in the environment and allowing the team to test procedures for hosting and running training events.
There are many specific examples of where research informed the HRD project. Given the importance that research-to-practice has in the role of the HRD scholar-practitioner, four examples are included in Table 1 to illustrate the extent and nature of the impact.
Examples of How Research Informed Practice.
The solution was evaluated in detail by the project team, both to demonstrate value to the company and also to determine whether the overall approach could offer benefits to other organizations.
The solution was assessed in terms of whether it met ABC’s needs for timely delivery, a positive learning environment, and cost-effectiveness. On timely delivery, the project was initiated in February 2010, with an investment decision made in April 2010. By June 2010, the virtual campus was operational and available for beta testing, with the campus hosting trial presentations 1 month later. By August 2010, the campus was hosting regular training courses that were transitioned from in-person; and within the first 12 months, over 500 employees created avatars and attended one of 20 2-day training courses or one of 23 1-hr presentations. Beyond that initial phase, between 2010 and 2012, over 1000 employees created avatars and attended one or more of the campus’ 50 multiday training courses as well as 45 shorter presentations.
The solution also met ABC’s need for a positive learning environment. Within the first 12 months, courses run on the campus were evaluated 4.66 out of 5.0 on student satisfaction (compared to 4.79 for in-person training), 4.41 for virtual training not held on the campus, and 4.22 for self-study e-learning. For one course that was held on both the campus and in-person, evaluations showed little difference in student satisfaction (4.70 for campus version, 4.77 for in-person). One of the standard courses taught on the campus applied an end-of-course test, and there was no difference in the average test score for students between the in-person tests taken prior to the move into the virtual environment and the virtual tests taken after the move. In addition, student feedback was broadly positive, and included such quotes as: “It was a fantastic experience”; “(The) presentation was significantly better than (other virtual formats) because between the chat function and the voice interaction, I absolutely couldn’t multitask on anything else”; “Training should be fun and this is a ‘fun’ way to deliver training”; “It’s more interesting than just listening to a phone call. It adds a visual element that feels more like you’re interacting with other people”; and “Felt a little more like a class room learning environment than normal virtual training. Like the text chat. Great way for people to provide answers to instructor questions.”
The solution also met ABC’s needs for cost-effectiveness. The solution was assessed in terms of a return on investment to the company (how the cost of creating and maintaining the environment compared to the savings for the company). The cost of creating the environment and maintaining it for the 1st year was US$52,000. The cost of employee salary involved in creating the environment, adapting the courses for the environment, launching the environment, and supporting the environment for its 1st year was US$110,000. This makes a total cost in Year 1 of US$162,000. Had the same courses been run in-person, the costs to the company would have been US$622,000 in the form of travel, hotels, training rooms, and equipment. This led to a Year 1 saving of US$460,000. In Year 2, the cost was US$50,000 compared to an estimated cost of US$550,000 had the courses been run in-person, leading to a saving of US$500,000. Over the first 2 years, the project cost US$212,000 for a saving of US$960,000.
The evaluation results were used to improve the student experience after the environment was released. For example, improved documentation was made available to students, and technical improvements were made. In addition, the evaluation results were used to justify the business case for a move from Second LifeTM into a more enterprise-friendly 3D virtual environment in late-2011; followed by justifying moving a further US$250,000 a year of training from in-person into 3D starting March 2013.
Implications
The approach taken in this case study had been attempted by very few organizations, and most who have attempted it shared only minimal information about the specifics of their approach and costs. The solution adopted by ABC demonstrated that select multiday in-person training can be operationalized in a 3D virtual environment with minimal adaptation, and highlighted the potential for moving skills-based training into virtual environments, such as management development. As shown in Table 2, this work can be completed with less than 1,000 hr of effort for an initial release of an environment capable of hosting corporate training courses.
Project Timeline and Resources.
This case study provides information for HRD practitioners on how to use 3D environments as a supplement to more traditional training tools. To assist HRD practitioners, the seven key lessons learned have been summarized in the following bullets:
Lesson 1—start with reality. In particular, ABC found that designing a 3D campus that looks like a real world campus reduced the challenges that employees faced when first using that 3D environment. Once employees had gained experience and become more comfortable in the environment, an organization could then push at the boundaries of creativity in the environment and how it works. As Kapp and O’Driscoll (2010) recommended, begin with the familiar as entering a 3D environment can be disorienting, and so it helps employees to create the 3D environment to look like something to which they can associate (in this case, a campus).
Lesson 2—provide support early and often. By providing orientation support to employees when they first used the environment, ABC eased anxieties and reduced problems associated with learning how to use and operate in a 3D environment. As Kapp and O’Driscoll (2010) recommended, focus on the first hour as employees’ opinions of the environment are formed quickly and can then be tough to change.
Lesson 3—generate champions at all levels. For ABC, this included getting senior stakeholders on board as early adopters by allowing private opportunities to try out the technology prior to it being used more widely. Beyond these senior stakeholders, as Kapp and O’Driscoll (2010) recommended, build a grassroots community by identifying passionate believers early on, and by building on their passion, motivation, time, and energy.
Lesson 4—take advantage of the environment to go beyond what is possible in-person. Part of the success of 3D virtual environments for training comes from them holding students attention for longer and encouraging them to behave the way they would in an in-person environment. However, a 3D virtual campus can provide functionality that cannot exist in-person, for example, students can discuss their questions and reactions in a chat window while the instructor is instructing, allowing students to help each other. The student and the instructor can both be helped by working in an environment that allows for comparable interactivity to an in-person setting, and then going beyond that to supplement it with additional possibilities for interaction. This can be achieved by adapting in-person training designs to run in a 3D environment, although ABC found that little adaptation was needed in most cases.
Lesson 5—allow for differences within the student population. Going into the project, ABC expected to see differences by generation and culture. However, in practice, fewer differences than expected were experienced. Admittedly, within ABC, employees using the campus were more likely to be in the 25 to 35 age range and were more likely to be technically savvy. However, the users included 20 to 60-year-old employees, including some that did not have a strong background in technology; and there was no evidence that those in older generations responded less positively. This challenged the belief that such technology only works for younger generations. In addition, ABC found that a 3D virtual training environment can reduce some of the traditional cultural differences seen in in-person training rooms. One example was the observation that employees in SE Asia were more likely to volunteer for role-plays when the course was run virtually than when run in-person, and were also more likely to ask questions and discuss personal development areas.
Lesson 6—instructors who perform better than others in an in-person environment are not necessarily those that perform better in a 3D virtual environment. ABC experienced examples of instructors being clearly better when in a 3D environment, and others where they clearly performed less well than in in-person training delivery. This has implications for how instructors are prepared for a 3D environment and also who is selected to instruct in such an environment.
Lesson 7—expect challenges along the way. The main challenges experienced by ABC were technical and related to stakeholder reactions. On technical challenges, the technology used to implement a 3D virtual training environment is still in its infancy, and adopting such leading edge technology is not problem-free. During implementation, ABC experienced challenges with bandwidth usage, video cards in laptops not being compatible, and access challenges with the technology blocked by firewalls at some locations. Adopting this technology is not easy, and needs to be viewed as an exploration into an emerging format. Challenges related to stakeholder reactions included needing to manage initial beliefs that 3D environments were not sufficiently professional for corporate training (addressed through providing these stakeholders with early access to the environment), or that 3D environments would not deliver an experience to match in-person training (addressed through sharing case studies from other organizations supported by early evaluation scores from ABC’s own use of the 3D environment).
These and other lessons have been disseminated via events run for practitioners, scholar-practitioners and HRD researchers, including a preconference at the 2011 AHRD conference in the Americas, a special interest group webinar run for the Virtual HRD SIG of AHRD, guest speaker sessions for HRD classes at universities, and chapter meetings of practitioner organizations.
The case described in this article also demonstrates some of the opportunities and challenges associated with acting as a scholar-practitioner. HRD scholar-practitioners seek to ground their practice in research and theory (Short & Shindell, 2009). In this case, there were relatively few articles on 3D environments, and few dealt specifically with training in such an environment. However, the case does show how the practice was informed by literature, and benefited as a result. As in other situations, part of the challenge for the scholar-practitioner is in how to identify and access the literature, how to interpret the literature within the context of a specific organizational situation, and how to combine literature with other sources of information to guide the design of an intervention.
HRD scholar-practitioners also seek to champion research and theory in the workplace (Short & Shindell, 2009). The speed with which most organizations operate makes it challenging to identify literature quickly and to summarize it in a way that it can influence practice. This is helped greatly by published literature reviews. However, in situations where practice is emerging and is leading research (as in the situation described in this case), the scholar-practitioner is unlikely to find detailed literature reviews or meta-analyses. Instead, the scholar-practitioner needs to be able to review articles and other sources of information, and quickly reach a decision about key themes and relevance.
The case also demonstrates key differences between the research completed by an academic researcher and research completed by a practitioner. In this case, evaluation scores demonstrate the performance of the 3D environment and its impact on student satisfaction and learning. That evidence is sufficient for organizational decision making (e.g., on funding future 3D environments), but lacks the rigor needed for scholarly research. This is an example of where a partnership between the practitioner and a researcher could make this setting available for scholarly research, and then work together to publish collaborative findings in academic journals.
HRD scholar-practitioners act as a bridge between research and practice in seeking further development of the field of HRD (Short & Shindell, 2009). In this case, the scholar-practitioner ensured that the approach and lessons learned were fed back to HRD practitioners and researchers. This was done via academic conferences as well as chapter meetings of practitioner organizations. Doing this requires that the scholar-practitioner adapts the messaging for the different audiences, and demonstrates how the HRD scholar-practitioner can bridge the worlds of academia and practice by describing how research is used in practice, and how practice can inform research decisions.
In summary, the case described in this article illustrates the opportunities and benefits arising from a scholar-practitioner approach to an HRD problem, and shows how the outcome can be informed by literature. At the same time, the case also illustrates the challenges of accessing literature, of practice being ahead of research, and of academic and practitioner research having different standards of rigor. The HRD scholar-practitioner routinely operates in spaces like this.
Critical Thinking: Moving Beyond the Case
In the section below, the author has moved beyond the boundaries of this unique case to challenge you, as the reader, to think more critically about the problem and solution they presented. Specifically, within the reflection questions, you are encouraged to think deeply about how this case, and others like them, could affect your work as a scholar, scholar-practitioner, or practitioner. Take time to answer these questions from your own perspective or with others in small group to explore a diversity of ideas and applications.
How can researchers and scholar-practitioners work together to reduce the time lag between a new practice appearing and the publishing of research based on that practice?
How can researchers and scholar-practitioners work together to increase the availability of literature reviews and meta-analyses, and in so doing increase the speed with which research can inform practice in an organizational setting?
How can HRD education better prepare students to operate in practitioner environments where practice is ahead of research, where decisions need to be made based on a quick read of available literature, and where models and theories have to be adapted on-the-fly to suit a specific organizational context?
In situations like the one described in the case study, how can scholar-practitioners judge the utility of practices in other organizations, recommendations from vendors, guidance from researchers, and other sources of information, and then determine a best way forward for the organization? For example, what if practices in other organizations appear to work well but are contradicted by literature?
In fast changing areas like technology, how can researchers and practitioners work together by ensuring that researchers have access to organizations as research sites and that research results are shared quickly before the technology changes and makes the research less useful? In addition, what are the implications for HRD practitioners given that delivery formats are changing so much and so often (e.g., the emergence of e-learning, the emergence of 3D virtual worlds and gamification, the emphasis on blended learning, and beyond these to new formats being developed in the coming years)?
Footnotes
Author Note:
This article was subjected to a two-tier, blind review process that did not involve any of the contributing authors who are currently members of the editorial.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
