Abstract
Affordance-based design (ABD) plays an important role in identifying interactions, especially effortless ones, between users and artifacts. Cognitive ergonomics extends our understanding of this effortless interaction. This study combines the two design methodologies together in order to reduce cognitive friction in using digital products. The design process of a compact digital camera is selected as a case study that includes the design of the physical shape for a camera and of its user interface. In designing a product shape, a design toolbox was developed that integrated a modified multi-objective genetic algorithm and the ABD, which was named as affordance-based interactive genetic algorithm. Using this toolbox and interactive user feedback, the camera design evolves toward a product that better satisfies the users. User interfaces (UIs) including linear and elliptic layouts were subsequently designed based on cognitive ergonomics. A predictive tool of UI, the Cog Tool, was used to evaluate the performance of skilled users on a given task by correlating the overall task completion time. Finally, this research has the potential to not only effectively address the shortcomings of the design of consumer electronics but also enrich the generation of design solutions during the preliminary design phase of such products.
Introduction
As the pace of technological change accelerates exponentially, the number of smaller, cheaper, and more powerful chips embedded in modern products such as household appliances and vehicles increases as well. These products are very often hybrid products, and the way the users interface with them is through their physical body and a graphical user interface (GUI). Compared to purely mechanical products, these electronic or electromechanical products have had a more significant impact on society than purely mechanical products because, in part, they exceed various human abilities such as memory, speed of operation, access to information, and information storage. However, since many of these aspects are invisible, product appearance is an important characteristic, persuading a user to purchase it, while for a designer, appearance through the physical body of the product serves mainly as its enclosure and user interface. The relationship between “form or structure” and “function” is thus often weakened. As a result, consumers experience difficulties using such products, a situation that increasingly widens with the development of technology. In other words, the distance, or what can be called the cognitive gap (Nicolas & Liora, 1994), between the consumer’s understanding of how to use a product and how the designer designed it to be used, especially if it has multiple functions, widens. Thus, at the very least, the users do not know how to interact with the product or are confused and possibly apprehensive when they see it, or worse they have a bad experience with a product they purchased and reject the technology. This phenomenon is defined by Alan Cooper as “Cognitive Friction (CF)” (Cooper, 2004), It is the resistance encountered by a human intellect when it engages with a complex system of rules that change as the problem changes, this term is a forensic tool.” As you interact with a user interface, it may change such as on a graphical user interface. In other words, cognitive friction occurs when a user is confronted with an interface or affordance that appears to be intuitive but delivers unexpected results. This mismatch between the outcome of an action and the expected result will cause users frustration and will impair the user experience.
Compared with the operational clarity and the simplicity of mechanical products performing a single function, users, especially older ones, often feel helpless using contemporary products. Furthermore, the level of education and the cultural background become important factors impacting the ability of users to operate electronic products (Kim & Christiaans, 2016). Norman has described this phenomenon with a story about the experience of one couple. Although both have a strong educational background (the husband is an engineering psychologist, and his wife is a physician), they are reluctant to use their washer-dryer combination: The woman can only remember a few settings, ignoring the rest while her husband is unwilling to come near the machine, let alone use it (Norman, 2013). The poorly designed interface could be what Cooper terms cognitive friction.
Cognitive friction is a recent phenomenon because the electronic products of today differ significantly from earlier and current mechanical-based products whose form conveys more clearly their function. For example, a steering wheel is used for changing the direction of a car: the car turns right or left depending on if a user rotates the steering wheel clockwise or counter-clockwise, and the shape of the steering wheel conveys the idea of rotation because of its round shape. To address cognitive friction, today’s designers use various approaches or sciences to make the product’s usage understood. Ergonomics, for instance, is an applied science focused on the physical and mental aspects of interacting with an object to ensure that this interaction is efficient and safe (Kroemer, 2001). From the perspective of physiology, ergonomics is used to gather data about human beings’ physical characteristics, specifically sizes and shapes. The size and shape of a product should then be designed to be appropriate for a human’s physical characteristics and should hint at how the artifact should be used (Lowe Brian et al., 2019; Shi-qing et al., 2018). Correspondingly, from the mental aspect, cognitive ergonomics focuses on reducing the users’ mental workload by designing artifacts that fit human cognitive competencies and needs (Gcvd, 2008). In Kansei engineering, for example, the relationship between the properties of a product and the feelings of users is explored from a psychological point of view (Mitsuo, 1995). In addition, product semantics and product semiotics consider design as a process of “coding by the designer and decoding by the users” (Krippendorff and Butter, 1984), an approach that focuses on the products conveying meaning and symbolism. In the design for environment (DFE), De Napoli et al. proposed an integrated model for the environmental assessment of industrial products during the early phases of design, which can help designers to select better product concepts (Luigi et al., 2017).
The approaches mentioned have different emphases for enhancing the users’ experience with their products. Few researchers consider the product and user as an integrated or co-dependent system (Noble Charles & Minu, 2010). Affordance can help bridge the gap between the product- and human-centered perspectives (Dhouha & Houcine, 2020). However, we have not found any published research centered on simultaneously designing a products’ physical body and user interface. To fill this gap and to reduce the cognitive friction, our study investigates the design of an electronic product with an emphasis on both the above aspects. We look into the relationship between affordances and the design parameters of the physical body as well as what pattern of layouts fits the interface constrained by the physical enclosure. From this research, there are some direct relationships between specific design parameters and affordances that can be then used in the design process. Also, a new GUI layout is proposed to replace the traditional ones and users efficiently find out what they want to achieve. From the above point of view, this work provides a possible approach to reduce the CF problem during design.
Background
Affordance
The term affordance was coined by (Gibson, 1977), who argued that an artifact contains clues and that users simply perceive how to use it: that is, what it affords the user for good or for ill. He emphasized that humans or “animal” perceive these physical properties directly, reducing the cognitive gap between the products and its users. This concept was quickly adopted in Engineering Design, Human-Computer Interaction (HCI), and Industrial Design (ID). In Engineering Design, Fadel and his students are among the pioneers in the application of the concept into this field, classifying affordance into three types based on the entities that are interacting: Artifact-User Affordances (AUAs), Artifact-Artifact Affordances (AAAs) (Maier and Fadel, 2009a), and Artifact-Environment Affordances (AEAs) (Hu & Fadel, 2012). In the fields of HCI and ID, affordance has become one of the design principles, along with others such as discoverability, feedback, conceptual model, signifiers, mappings, and constraints (Norman, 2013).
According to Norman, there is abundant symbolic information in physical reality, that is, affordances. For example, a genuine hammer or a mug provides many physical clues about its purpose and how it should be used, information that users can directly and easily process. However, for electronic devices, consumers have to learn and remember the complicated operation process when they operate a product for the first time. The lack of physical characteristics to help users identify affordances is the primary difference between traditional mechanical products and consumer electronics. This, in turn, is the main cause of the increasing perception gap between users and products, somewhat contradicting Norman’s contention (Norman & Donald, 2002) that the perception of a product’s affordances is an important factor in its success. But the development of affordance is also accompanied by the challenge of how to apply this concept to design.
According to Gibson, people could understand how to operate a product through its visual characteristics or properties without any need for symbols or text. For contemporary products, Gibson’s concept appears idealistic as, for example, it is especially difficult to describe the interaction between humans and aspects of a graphical user interface (GUI). Based on Gibson’s concept, Djajadiningrat designed a video deck, eliminating the common symbols and color signs in his model (Djajadiningrat, 1998). He found that affordances provide a variety of ways for interaction, but cannot enhance a products’ usability (ease of use). Norman was one of the pioneers in applying affordances to design artifacts, increasing the applicability and practicability by dividing the concept into real and perceived affordances. El Amri proposed that if at least one of a product’s perceived affordances is good, users need less help in understanding the usability of this product, and its adoption would be enhanced (Dhouha, 2019). Thus, perceived affordance is important to designers. As Norman writes in his paper: “The designer cares more about what actions the user perceives to be possible than what is true. Moreover, affordances, both real and perceived, play very different roles in physical products than they do in the world of screen-based products. In the latter case, affordances play a relatively minor role: cultural conventions are much more important.” (Norman, 1999). He further illustrates this idea about a screen-based product, explaining that a circle drawn by a designer on a screen to indicate where a person should touch is not an affordance but rather a signifier as defined in the new edition of The Design of Everyday Things (DOET) (Norman, 2013).
In general, many electronic products are similar in shape: a closed and seemingly meaningless box. While the components inside the box may be fast and powerful, users do not interact with them directly. They interact through a GUI, relying on their knowledge and experience, interactions that involve a significant amount of cognitive knowledge. Researchers have addressed this situation, for example, Hartson defines and uses four complementary types of affordances in the context of interaction design and evaluation: cognitive affordance, physical affordance, sensory affordance, and functional affordance (Rex, 2003). Besides, Nagy and Neff proposed “imagined affordance” to describe three relationships, such as users’ perception, attitude and expectation, the materiality and functionality of technologies, and designers’ intentions and perceptions (Peter and Gina, 2015). In order to build and validate a measurement scale of perceived design affordance in high-tech products, El Amri and Akrout proposed “symbolic affordance” in their Perception Design Affordance Scale (PDAS) (Dhouha & Houcine, 2020). However, there remains a lack of agreement on terminology, for example, between cognitive affordances, conventional affordances and signifiers. This paper focuses on the complementary relationship between affordances and cognitive ergonomics using Norman’s concept of perceived affordance as exemplified by a chair that affords support both for sitting and carrying.
Cognitive ergonomics
Cognitive ergonomics is a branch of ergonomics based on cognitive psychology and other related sciences. Therefore, it shares a common purpose with ergonomics, which aims to facilitate human performance through the adaptation of tools or devices to human characteristics and preferences. Further, it focuses on the human mental processing that is involved in processing information, such as rehearsing and recalling. While cognitive psychology establishes general theories about mental behavior, cognitive ergonomics aims to apply this knowledge to make the human–product interaction compatible with human cognitive abilities and limitations (Haan, 2010), meaning it focuses on the application of cognition to a product.
Application of cognitive ergonomics in user interface design
User interface design, while not new, is an essential part of product design (Mariia & Angela, 2019). Today, increasing numbers of screens and graphical user interfaces are embedded in products. Therefore, determining what information should be displayed on a screen or which layout matches the users’ mental model is certainly worth exploring. The purpose of cognitive ergonomics is to develop and adapt cognitive devices and their uses to improve human information processing to increase efficiency, reduce errors and accidents, and increase well-being (Haan, 2010).
Cognitive ergonomics may help to improve the performance of a user interface by addressing, for instance, the layout of icons. A careful layout of icons plays a critical role in clarifying the functions of electronic products, and this is helpful for the users to understand and interact with the products. Vicente developed design principles for the layout of complex interfaces based on skills and knowledge, an approach referred to as Ecological Interface Design (EID) (Vicente Kim, 2002). In further research, some scholars proposed guidelines on how to design large screen displays, focusing on a digital interface based on a Process Control System (PCS) (Braseth & Ritsland, 2013). In addition, Nielsen found that based on eye tracking, users read web pages in an F-shaped pattern, with the upper part of the content area being noticed first and attracting the most attention (Nilsen, 2017). This F-layout, seen in Figure 1(a), is the natural behavior of most web surfers. Other patterns have also been found, including the Gutenberg diagram, the Z-pattern layout, the Zig-zag pattern layout (see Figure 1(b)), and the Golden-triangle layout. Traditional layouts: linear layouts (a) F-shaped pattern (b) Zig-zag pattern.
It can be found that most researchers focus on the layout of complex, large interfaces and web pages. However, with the increased need to provide visual information for all kinds of devices such as power drills, handheld medical devices, and a variety of consumer electronics, increasingly smaller screens are being designed. Compared with large, complex screen-based interfaces, there is just a few published research papers guiding the designer on how to design these handheld digital devices’ (smaller) interfaces or how to standardize them, an area where research is much needed because these products are designed for users who neither have enough time to operate them nor want to be trained on how to interact with them. Cognitive friction is apparent when users interact with these products. To address this specific concern, this research focuses on the following issues: • What type of display layout is suitable for handheld digital device’s screen? • How can a handheld device interface be designed based on cognitive ergonomics?
Research purpose
The purpose of this study is to address the issue of cognitive friction. As we mentioned in the introduction, cognitive friction occurs because of a non-existing affordance or a mismatch between intended affordance and physical shape. In this research, a compact digital camera is used as a case study as it has the typical characteristics of consumer electronics: a short life cycle, rapid updating, a unified shape and a diversification of user interface displays, the latter two primarily causing the issue of cognitive friction. Furthermore, a camera is a handheld tool, which involves a significant number of Artifact-User Affordances (AUAs).
In order to provide a clear perception of affordances for users, an Affordance Based Interactive Genetic Algorithm (ABIGA), a tool that combines the theory of affordance and a modified multi-objective Genetic Algorithm is used in the initial steps of the design of a new camera. Using ABIGA and users’ input, the body and screen size of a camera can be “optimized.” The screen size is also the dimension of the user interface. From the mapping relationship between designer-specified affordances and design parameters that the users generate, designers can strengthen intended affordances during the design process by modifying the enclosure appropriately. Once the screen size is identified, the user interface can be designed based on cognitive ergonomics (CE) and evaluated by the Cog Tool.
ABIGA was developed and used in this research for evolving the product’s form since this tool is designed to optimize a product’s geometry based on users assessing the quality of various affordances. Multiple user-centered virtual prototypes were generated, models that then inspired designers in developing the details of the product. Simultaneously, cognitive ergonomics was used to emphasize the cognitive aspect of the product, specifically, its graphical user interface (GUI). Then, the software Cog Tool (John Bonnie et al., 2012), a predictive tool of quality of a GUI, was used to compare designs and identify user interface (UI) differences based on a timeline visualization. The Cog Tool can determine which layout is more in accordance with the users’ mental model. From this part of the research, a new pattern layout is proposed for handheld digital devices.
A broad overview of the design process is shown in Figure 2. A broad overview of this research.
To eliminate other interference factors, this research has the following limitations: 1. It focuses on a compact digital camera rather than some other type (film camera, SLR, or DSLR) because the SLR/ DSLR design considers advanced users as the target population, consumers who address the issue of cognitive friction through professional training. 2. ABIGA focuses on the relationship between affordances and the camera’s shape. Other aspects such as material, color, and the layout of the GUI are not considered in ABIGA, but these design elements are taken into account later using cognitive engineering.
Product form evolution with ABIGA
Introduction to ABIGA
According to Maier and Fadel, one of the properties of affordance is form dependence (Maier et al., 2007, 2009b), meaning specific affordances may vary with changes in the design parameters. ABIGA is also created based on the form dependence property of affordances. In addition, ABIGA considers the product and users as an entire system. This tool is a design assistant system specifically for the early stages of the design process as it attempts to progressively improve a product’s form based on the perceived affordance qualities observed by users. The search mechanism used is AMGA2, the second-generation Archive Based Micro Genetic Algorithm (Santosh et al., 2011), which provides an efficient way to evolve design solutions using a very small population by taking advantage of an archive (Santosh et al., 2011). In this research, ABIGA was used with the aim of reducing the cognitive friction between the users and the product, based on their feedback.
Preparing for the implementation of ABIGA
Affordances and design parameters chosen
In designing a compact digital camera, the affordances of interest to the designer and the explanations provided to the users are the following: • Portability: Interaction between the users and the camera when they carry it or store it in a bag/pocket. • Hold-ability: Holding interaction between the user and the camera using either one hand or two. • Exposure-ability: The unit of measurement of the total amount of light permitted to reach the electronic sensor while taking a photograph. • Screen view-ability: Visual interaction between the users and the camera that allows them to conveniently see photos on the screen.
The six design parameters used to determine the body’s basic features and their ranges are shown in Figure 3. More detailed information about the affordances and design parameters to be chosen and how to choose them was provided by Chen et al. (2019). Design parameters.
User interface and participants
Detailed information about the software ABIGA, including the development of the code, its use in various case studies, and its limitations can be found in Mata and Chen (Chen et al., 2019; Ivan et al., 2018). The web evaluation page for the camera based on ABIGA is divided into four sections as seen in Figure 4. At the top of the page, a short paragraph introduces the experiment and explains how to use the software. The camera is shown on the left side, the representation consisting of three views: front, top, and back. Size reference for the camera is shown using a hand holding an AA battery. This hand is a 50th percentile representation obtained by averaging the size of the typical male and female hand. The breadth and length of this hand are 88.4 mm and 174.9 mm, respectively (DTI, 2002). The evaluation survey rates the quality of the affordance on a 7-point Likert scale using quality values ranging from −3 to +3. Negative values indicate that the affordance exhibits poor usability; positive, good; and zero indicates a neutral rating. Users select each affordance quality value on a slider. Camera user interface for evaluation.
Participants rated the quality of the affordances of this camera presented to them, with most respondents being mechanical engineering students, although the application is on the web and can be accessed by anyone. Based on the data obtained after the completion of the experiment, five users with ages ranging between 24 and 29 completed all the evaluations.
Evolution of designing camera form
The evolution of designing a camera
Although ABIGA as a multiple-objective Genetic Algorithm (GA) does not combine objectives, to show the overall evolution of the camera, the sums of the affordance over each generation were plotted to display the solution’s overall quality evolution (Ivan et al., 2018). The interactive aspect is related to the capture of the users’ feedback as to the value of each affordance. According to Banerjee et al., 15–20 generations/iterations can achieve satisfactory results using IGAs (Amit et al., 2008). Therefore, this camera study was run for 20 generations. As shown in Figure 5, the concepts tend to improve as the generations increase based on the evolution of the average fitness of the products’ affordances, meaning that the GA is creating solutions of better perceived quality overall (Table 1). Camera: Average fitness of population versus generation. The range of values.
Relationship between affordance and design parameters
Binary categorization of user response.
Binary logistic regression p-value results.
DP = design parameters; Aff = affordances.
The relationships identified in Table 3 provide the designers with an indication of the impact of the parameter(s) on the product’s perceived usability. For instance, if the designers want to improve hold-ability, the width of the camera seems irrelevant to the users, but its height and possibly its fillet radius are two parameters to modify. However, at times, there is no apparent direct relationship between a parameter and the affordances being considered. For example, screen size view-ability depends on screen size, which indirectly depends on the camera’s width, height, and fillet radius since these parameters constrain it. Two of these relationships in the camera study reported here are analyzed and discussed more fully in the following sections.
As listed in Table 3, the exposure-ability strongly relates to the camera height and the lens size. The diameter of the camera’s lens is constrained by the height of the camera because the lens diameter should be smaller than the camera’s height which is smaller than its width. In order to have high exposure-ability, larger lens size is expected, which can explain why larger lenses are continually being developed, one such example is the SLR camera (Single Lens Reflex camera). Figure 6 demonstrates the fitted curve from the binary logistic regression analysis indicating the relationship between the lens size and exposure-ability of the camera. It can be found that the design parameter lens size is significantly associated with the ability of the camera to capture light for an effective photograph, that is, its exposure-ability. Logistic regression: Exposure-ability versus lens size.
In Figure 6, the data points represent the users’ binary categorized responses, with the curve representing the increased probability of a positive assessment as the lens size increases. For example, for a lens size of 0.6 (∼38 mm), the probability that a user would perceive the exposure-ability of the camera as positive is 0%, while lens sizes bigger than 0.9 (57 mm) have a positive assessment from almost 100% of the users.
Logistic Regression also identified the relationships between hold-ability and both height and fillet radius. The results show that the hold-ability has a positive effect on height, but a negative correlation with fillet radius. The designer needs to attempt to understand the reason for these relationships.
According to the Occupational Safety & Health Administration (OSHA), a hand in the shape of a “C” suggests holding or griping (Figure 7(a)). The suggested size for these hand postures is between 44.45 mm and 76.2 mm (Figure 7), within the constraints for the height of the camera in this study of between 40 mm and 79 mm. In the experiment, the user input obtained indicated that the participants prefer holding the camera as indicated in Figure 7 (a) or (b) when they perceive that the height is small. When the height is perceived to become larger, we assume that the posture of their hand changes, and they hold the camera as indicated in Figure 7c. This latter position does not require the exertion of excessive force by the users, explaining why users indicated that a bigger height size is easy to hold. Three postures for holding.
This study also found that the fillet radius has a negative effect on hold-ability. Some users may think that a large fillet radius makes the camera easier to hold since there are no sharp corners to hurt them. But this larger fillet radius also has the potential of increasing slip-ability.
Subset of best solutions for the product form
Two design solutions used in this research (unit: mm).
Once the physical dimensions of the camera are identified, the information on the screen, which is the primary interface for the user, has to be properly designed. This research considered this aspect of the design using cognitive ergonomics.
UI design based on cognitive ergonomics
Introduction to the Cog Tool
This research used the Cog Tool, which constructs valid Keystroke-Level Models (KLMs) (Card Stuart et al., 1980), to estimate the time of completion of certain tasks rather than another tool for three reasons. First, the Cog Tool evaluates UI design ideas quantitatively before investing resources in programming those ideas (John, 2010). This predictive human performance modeling has been proven to reliably and quantitatively predict various aspects of usability (Brinck et al., 2002; John Bonnie et al., 2012). In addition, it can also provide design recommendations by displaying timelines showing how long a user would look, think, and act while accomplishing a task. Designers can then identify differences between UIs by comparing their respective timelines (John Bonnie et al., 2012). Finally, this tool can save time and money as it does not require real end-user involvement. Thus, it is an appropriate initial measurement tool for evaluating the performance of the two UIs designed in this research.
UI design
According to McKay, the scan path for large displays is usually an arc beginning in the upper left and ending in the lower right, meaning the layout is understandable, simple, efficient, and orderly (Mckay, 2013). However, this is not the case for handheld digital devices (small) displays such as the ones being considered here. For smaller interfaces, the screen center is also the visual center, meaning that this area draws the most attention from the users and is the first one they focus on. Therefore, a new pattern layout is proposed here, as seen in Figure 8. The number one locates the center square, which will display the most important information or information that needs to be confirmed. Nontraditional radiant layout.
For this reason, an elliptical design is drawn, which is more centric than the ones used for large screens, thus fitting the rectangular shape of a small screen. The purpose of this layout is to guide the users’ scan path from the center of the screen outward in all directions as seen in Figure 8 and Figure 9. The effectiveness of this elliptical layout will be compared with the traditional layout in the Evaluation Section. This research, thus, developed two layouts, a traditional linear layout, which simulated the actual interface of the Cannon SD1300 and was used as reference (Figure 10), and an elliptical one developed based on the concept that the center attracts more user attention. Elliptic layout of the camera’s interface. Linear layout of the camera’s interface (as reference).

The UI elements seen in applications include both interactive ones such as buttons and informative elements such as images, and color. In this research, the icons and buttons are the same for the two layouts for consistency, thus eliminating the effect of these factors on the results.
Icon design aims to facilitate communication and understanding with the users, ensuring the interaction with the UI requires minimum cognitive resources. Two design approaches, metaphoric and realistic, both from cognitive psychology, are used to provide hints to the users about how to interact with the texts and icons. For example, the image playback icon uses a circle to imply that users can browse through their photos. The circle icon and the behavior of photo browsing share the same characteristics, that is, users have the freedom to choose any photos that they want to check, but these photos are organized in the order they were taken. Other icons, such as for printing, deleting photos, and switching the settings, are designed in the same way. They could imply their operation or destination. Users will understand the meaning of these icons based on their experiences in the real world. In addition, based on cognitive ergonomics, other aids can be used to reduce the cognitive friction between users and the UI, including shading, underlined texts, arrows, and highlighted texts.
In this research, color is used not only to reflect the UI’s aesthetics, but also to indicate icon states through. For example, to draw users’ attention, being clicked icons or buttons are displayed in highlighted colors, with orange and blue used as target or foreground colors in these instances. The icons or buttons that are not clicked are displayed in light gray as seen in Figure 9 and Figure 10. In addition, a camera is usually used outdoors, so high contrasting colors are needed to ensure visual clarity. Based on research from cognitive ergonomics, white/ black and yellow/ black have been found to be the most effective combination (Shieh and Lai, 2008). Further, Pastoor found that dark colors function better as background and light ones as the target color (Pastoor, 1990). To enhance the visibility of the UI developed here, black and dark gray was selected as the background colors and blue and orange as the target colors as seen in Figures 9 and 10.
Evaluation
The tasks set-up in the shooting mode.
Once the tasks are set, we can use the Cog Tool (John Bonnie et al., 2012) to make quantitative prediction of human performance. Project, task, design and frame are mainly included in Cog Tool. The more detailed information about how to use the Cog Tool was provided by http://cogtool.hcii.cs.cmu.edu/
Figure 11 presents the predicted time results for human performance. It can be found that skilled users will spend longer operation time on the Canon linear UI than that on the elliptic one in completing all of the three tasks. The simulation result demonstrates that the elliptic UI is able to increase user efficiency and reduce the cognitive friction. Results from the two layouts for the same tasks.
To analyze the differences between the two interfaces and user interaction in more depth, this study not only focused on task completion time but also emphasized the cognitive process by comparing the timelines obtained from the Cog Tool as seen in Figure 12. For designers, measuring only the completion time of a task does not completely identify UI problems or provide recommendations for optimization or redesigning. The timeline visualizations from the Cog Tool show what users see, think, and do. Comparison of the visualization results from the two interfaces.
Figure 13 shows the series of procedures for the interactive behavior needed to be completed for Task 1. The procedure involves hand and eye movements, some of which are parallel, for example, the movement of the cursor to the target point and eye movement to the target simultaneously happened. The visualization results found that the time difference between the two UI’s layouts is primarily influenced by three movements: moving the cursor to the target point, the first eye movement to the target point, and the second eye movement to the target point. Results from the different layouts for the same task (a) linear layout (b) elliptical layout.
To determine the influence of the distance between the button and the target on the predicted time cost by eye movement and cursor movement, the target point was simplified as point 3 in the visualization modeling as seen in Figure 14. Points 1 and 2 stand for the button position. The results show that the cost time for the first eye movement has a positive relationship with the distance between the starting point and the target point for both linear and elliptical UI layouts. However, there is no relationship between the cost time of the cursor movement and the distance. Linear regression was used to verify this relationship between the first eye movement and the distance as seen in Figure 15. A similar conclusion was found in Wang et al.’s study (Wang et al., 2015). Based on these results, it was concluded that the first eye movement helps to find the target points. The coordinates for two different layouts. Linear relationship between time and distance.

These results for the elliptical layout proposed here appear to contradict with the visual workflow from left to right and from top to bottom in the traditional layout that requires fewer cognitive resources for mastering interface information. Previous research focused on large-screen interfaces while this study focused on small screens where the interface is inside the range of sight. For this reason, the crossover visual workflow of the elliptical layout (Figure 14 (b)) does not affect the information processing of the user.
Besides analyzing the time cost for the first eye movement to the target point, the similar method was also used to identify and analyze the spent time for the second eye movement to the target point. The analysis results indicate no direct relationship between the distance and the time spent on the second eye movement to the target point. Additional factors affecting this relationship will be investigated in future work.
Through the above analysis, it was concluded that users will think, observe, and look for the desired target when they interact with interfaces. In this process, the first eye movement tries to find the target point. And the second eye movement plays the role of confirmation. From this perspective, different layouts should result in different time consumption. Using the situation investigated here as an example, the elliptical layout is considered advantageous since the distance between the starting point and the target is small, facilitating the user’s search for the target.
Integrating the interface and physical shape of the camera
Design plan A
The modified camera compared with its original version (unit: mm).
Design plan B
The design parameters for this camera’s physical body were also obtained from the ABIGA optimization as seen in Table 4. The width and height of the new camera are larger than the reference, but the depth is similar to the original. The screen size of the newly designed camera is 82 mm*48 mm. In other words, its screen is more rectangular. The newly designed camera is for business people, a population with more professional needs than the younger group. As we all know, this group focuses on product quality. Based on this knowledge, the new screen with large size can enhance the user experience of screen-viewing. According to this, we infer that the present screen size is wider with an aspect ratio of 21:9 instead of 16:9. Since they frequently carry briefcases, a wider camera still features portability for this group. As for hold-ability, these users prefer using it with two hands rather than one, meaning the height also can be increased. Because of this increase in size, the operational buttons are arranged on both sides of the screen, different from design A as seen in Table 6. This arrangement allows both hands to operate the camera, meaning this general design not only takes the right hand into consideration but also considers a left-handed operation. The lens and screen of this camera occupy a large proportion of the shape to optimize the exposure-ability and screen view-ability. To improve the users’ perception of hold-ability, a skid-resistant material such as abrasive plastic and a large chamfer on both sides is adopted. Finally, since this group is older and, thus, prefers a familiar user interface, the traditional linear UI is used as seen in Table 6.
Conclusions
To reduce the problem of CF, affordance-based design and cognitive ergonomics are conducted in a combined manner to design product. Focusing on a digital camera, this case study developed an ABIGA to evolve the product’s physical body, allowing end-users to evaluate its affordances. This feedback was used to optimize specific features. The first half of this paper showed that this tool can lead to improved affordances as the generations increase and the physical characteristics of the design evolve. In the second half of the paper, the results from the Cog Tool software tool predicted that users would spend less time using an elliptic radial design layout than the traditional unidirectional one. In other words, the elliptical layout possesses a cognitive advantage. This paper, thus, addresses how designers should consider affordances and cognitive friction to enhance the user experience with their products.
This research suggests that to avoid the gap between product and user, multiple design methods and processes should be integrated. More specifically, currently, ABIGA considers only the product’s structure and does not consider color, texture, and other design elements that also play important roles in narrowing the distance between a product and users. Future work needs to address these design elements. In addition, to some extent, the 2D representation may affect the user’s perception of a product’s affordances. Therefore, a 3D representation and an appropriate context to facilitate information processing should be explored in the future. Also, design parameters for structural elements should be considered in relation to a specific affordance. For example, a handle may afford a better hold-ability than a hand holding the body of the camera.
This research used the Cog Tool to consider the cognitive ergonomics of the camera, finding it is effective for designing a useable UI; however, since development in this field is not as well-developed as software engineering, this case study is based on limited knowledge. In the future, tools should be developed to create UIs based on structured, methodological knowledge. In addition, how to evaluate a hybrid product that combines a user interface and a physical body merits investigation. Future work could then improve these interfaces based on the visual feedback. For these limitations, the methodology that combines affordance and cognitive ergonomics is appropriate for hybrid digital products, such as handheld digital products. In the future, other products than handheld devices should be explored by the proposed methodology.
Most participants in the study are from the department of mechanical engineering. Therefore, the result of this study was largely swayed by the particular population surveyed. In a future implementation of ABIGA, the participants with different educational background will be accepted to broaden the validity of the results and see how users’ backgrounds may affect the design.
Footnotes
Acknowledgments
This research is supported by the National Natural Science Foundation of China (No. 52005251) and the Social Science Foundation of Jiangsu Province (No.20YSC011)
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The author received the financial support from National Natural Science Foundation of China (52005251) and Jiangsu Social Science Youth funded project (20YSC011).
