Cognitive Load Measurement Methods for Usability Testing: A Critical Analysis and Framework for Interface Evaluation

Mouse Dynamics

Mouse dynamics are one of the behavioural measures of cognitive load that are supported based on our human motor system. Mouse dynamics rely on different mouse movement attributes including speed, direction, action, distance, and time (Ahmed & Traore, 2007). Based on these attributes, higher level features such as acceleration, angular velocity, scroll wheel activity, clicks/double clicks, drag-and-drop operations, point-and-click operations, and silence can be measured (Ahmed & Traore, 2007; Gamboa & Fred 2004; Hocquet et al., 2004; Jorgensen & Yu 2011; Pusara & Brodley, 2004). These features are aggregated and measured in distinct time periods for analysis (Grimes & Valacich, 2015). There are different applications such as Mouse Tracker and Mousotron that can be used to measure mouse movement data.

Several studies showed that mouse dynamic attributes decrease with increasing cognitive load, since users have less resources to perform the tasks as load increases and so have less resources left to move the mouse (Grimes & Valacich, 2015; Khawaji et al., 2014; Rheem, Verma, & Becker, 2018). In particular, if the mouse movements are redundant to the task at hand, by increasing cognitive load the movement will be decreased. However, if the mouse movement is used to complete the main tasks, the movements may increase when increasing the complexity level of the task as participants require more effort to find the solution and complete the task, and the mouse movements may support this goal. Whether mouse movement increases or decreases as cognitive load changes will depend on the purpose of the movements to the task at hand. See Table 10 for relevant studies that used mouse dynamics in conjunction with other measurement methods.

Mouse-dynamic measures have been recognised as an effective method for assessing cognitive load and identifying usability-related difficulties. Across the reviewed studies, mouse-movement indicators were analysed alongside complementary measures such as task-performance duration, test-performance scores, usability questionnaires, and mental effort ratings (e.g., Darejeh, Marcus & Sweller, 2021, 2022; Fuller et al., 2020; Grimes & Valacich, 2015; Hibbeln et al., 2014; Khawaji et al., 2014; Kortum & Acemyan, 2016). Findings consistently showed that mouse-movement characteristics—such as speed, distance, and acceleration—varied systematically with task difficulty and aligned with subjective and performance-based indicators of cognitive load. Although mouse-dynamics were primarily applied to evaluate interface usability rather than to establish psychometric validity, the convergence observed across multiple measurement methods provides applied evidence of the convergent validity of mouse-dynamic measures for detecting cognitive load variations during interactive tasks.

Linguistic Features

One of the indirect subjective methods of measuring cognitive load that can be used in evaluating usability is analysing the complexity level of users’ spoken language including sentence length, number of words, punctuations, syllables, use of pauses, repetitive words, corrections, complexity, and the comprehensibility level of the words. Linguistic complexity is measured by two major factors known as syntactic complexity and semantic difficulty. Syntactic complexity observes the sentence length, which is the best indicator of language complexity. Semantic difficulty analyses the use of words, their lengths (syllables and letters) and their structure (Lennon & Burdick, 2004).

Different studies showed that there will be different linguistic patterns based on the complexity level of the task and the associated cognitive load. Some have indicated in high-load situations, participants’ speech rate, amplitude, speech energy, and variability will be increased (Brenner et al., 1985; Lively et al., 1993). Others have reported peak intonation and pitch range patterns (Kettebekov, 2004; Lively et al., 1993) and pitch variability (Wood et al., 2004) are related to high cognitive load. Also, by increasing the cognitive load level, participants may use significantly longer sentences with more complex structures that can make comprehension more difficult (Khawaja, Chen & Marcus, 2014). Khawaja, Chen and Marcus (2012) also found that as the task complexity increased, so teams collaborate more and their language patterns change as reflected by an increase in speech, use of longer sentences, more disagreement, and using more plural pronouns such as “we” and “they.”

Across these studies, the recording and analysis of language varied: some examined spoken utterances captured through audio recordings (Khawaja, Chen & Marcus, 2014; Oviatt, 2006), whereas others analysed team dialogues or typed linguistic inputs within interactive or simulation-based environments (Khawaja, Chen & Marcus, 2012; Tuček, Mount & Abbass, 2012). These methodological differences partly explain the variation in reported linguistic indicators of cognitive load. Since linguistic methods can measure the load in real time, it can be an efficient method to find different usability issues, especially if software users need to communicate and speak with each other. See Table 11 for relevant studies that used linguistic features alone or in conjunction with the other measurement methods.

The reviewed studies indicate that linguistic features can serve as effective indicators of cognitive load and usability. Linguistic analysis was used either independently or in conjunction with subjective rating questionnaires or performance measures (e.g., Khawaja, Chen & Marcus, 2012, 2014; Oviatt, 2006; Tuček, Mount & Abbass, 2012). Findings consistently showed that increases in task difficulty were associated with reduced speech intensity, higher pitch, and simpler sentence structures, reflecting the diversion of working-memory resources toward task processing. In collaborative contexts, greater cognitive load corresponded with more tightly coupled speech patterns and shared linguistic structures among participants. Although these studies primarily applied linguistic analysis to evaluate user performance rather than to establish formal psychometric validity, the consistent convergence with subjective and other behavioural indicators provides applied evidence of the convergent validity of linguistic features as markers of cognitive load in usability evaluation.

Eye Movements

Eye movement can be simply measured with the use of cameras and eye tracking devices without the need to attach anything to users that can make it uncomfortable for them and decrease reliability of the results. Different studies have proved the efficiency of measuring eye movements in the process of usability testing and designing adaptive e-learning systems (El Haddioui, 2019). Some researchers found that when self-report questionnaires cannot show an accurate result in the usability evaluation, eye tracking methods can be used as an alternative and reliable solution (Schiessl et al., 2003; Wang et al., 2020).

Compared to EEG or fNIRS, which capture direct neural activity associated with cognitive processing, eye-movement measures provide an indirect but highly interpretable reflection of visual attention and information-processing strategies. While EEG and fNIRS offer higher temporal precision and sensitivity to instantaneous cognitive fluctuations, eye-tracking methods are less intrusive, easier to implement in usability studies, and can reveal how users allocate attention across interface elements in real time. Accordingly, eye-movement data can be used in conjunction with EEG to triangulate cognitive load findings and enhance interpretive validity (Beatty, 1982; Baldwin & Cisler, 2017; Hess & Polt, 1964).

Eye movement measures can be categorised into behavioural (voluntary) including fixations and saccades and physiological (involuntary) including pupil dilation and blink rate that will be discussed in the physiological measures section (Chen et al., 2016; Pfleging et al., 2016; Rudmann, McConkie, & Zheng, 2003).

Fixation is a behavioural (voluntary) eye movement measurement which refers to a situation when the eyes remains still over a period of time, from 200 milliseconds to several seconds, focussed onto an area of interest (AOI). In this measurement, the interface element that is relevant for the current cognitive activity is identified based on the gaze direction (Rudmann, McConkie, & Zheng, 2003). Increasing the fixation duration on an interface element can show that users needed more attention as the understanding of the interface was difficult for them. It can be an indication of increased processing in working memory and high cognitive load (Chen et al., 2011). See Table 12 for relevant studies that used the fixations measurement method alone or in conjunction with other measurement methods.

The reviewed studies indicate that fixation-based eye-tracking measures are effective indicators of cognitive load and usability. Across investigations, fixation metrics were frequently analysed alongside other measures such as usability questionnaires (Huang & Zhang, 2022; Zardari, Hussain & Arain, 2021), task-performance duration (Calvo et al., 2022; Chynał et al., 2012; Wang et al., 2014; Zardari, Hussain & Arain, 2021), test-performance scores (Calvo et al., 2022; Guerberof, Moorkens & Brien, 2021; Zardari, Hussain & Arain, 2021), EEG (Cheng & Wei, 2018), and EDA (Bruneau et al., 2002). Findings consistently showed that fixation duration and frequency increased with higher task complexity and reduced usability, mirroring patterns observed in error rates and task-completion times. This alignment between visual-attention metrics and subjective, behavioural, and physiological indicators provides applied evidence of the convergent validity of fixation-based measures for evaluating cognitive load. Although most studies focused on applied usability assessment rather than psychometric validation, the consistent fixation-based findings across diverse interfaces underscore their robustness as indicators of attentional demand in usability contexts.

Another behavioural (voluntary) eye movement measurement involves the use of a saccade, which refers to rapid eye shifts between two locations (from one fixation to another) that reflect users’ visual search behaviour and attentional shifts. The most common measurement method for saccades is observing the patterns and velocity of scan paths, defined as the number and duration of gaze fixations across different AOIs. In usability contexts, increasing saccade velocity or frequency indicates higher cognitive load because users must exert greater mental effort to locate relevant interface elements, suggesting inefficiencies in visual layout or information organisation. When cognitive processing is smooth and the interface supports user expectations, saccades tend to be shorter and more targeted; when users struggle to find information, they become faster, more frequent, and less predictable. Conversely, a decrease in saccade velocity can be an indication of tiredness or reduced attentional engagement (Barrios et al., 2004; Chen et al., 2011). See Table 13 for relevant studies that used the saccades measurement method alone or in conjunction with other measurement methods.

In the reviewed studies, saccade-based eye-movement measures were recognised as effective indicators of cognitive load and usability. These measures were frequently examined alongside other indicators such as fixation metrics, task-performance duration, error rates, and EEG (Ehmke & Wilson, 2007; Keskin et al., 2020; Renshaw et al., 2003). Across these investigations, increased task complexity and reduced usability were associated with shorter, more frequent saccades and higher fixation counts, reflecting greater visual-search effort. The convergence between saccadic behaviour, performance outcomes, and physiological responses provides applied evidence of the convergent validity of saccade-based measures for detecting cognitive load variations, even though these studies primarily focused on interface evaluation rather than formal psychometric validation.

Pupil Dilation and Blink Rate

In the category of physiological cognitive load measures, analysing pupil dilation and blink rate are two of the most reliable approaches and are used by many, as they can be simply measured with the use of cameras and eye tracking devices without the need to attach anything to users that can make it uncomfortable for them and decrease reliability of the results.

As explained in the previous section, eye movement measures can be categorised into behavioural (voluntary) including fixations and saccades and physiological (involuntary) including pupil dilation and blink rate (Chen et al., 2016; Pfleging et al., 2016; Rudmann, McConkie, & Zheng, 2003). In the next sections the physiological measures of eye movement are discussed.

Pupil dilation is a physiological (involuntary) eye movement measurement that is affected by cognitive processes. With increased cognitive load, users’ pupils dilate and by decreasing the difficulty level of the task or towards the end of a task, pupils diameter size decreases (Chen et al., 2011; Klingner, Kumar, & Hanrahan, 2008; Porta, Ricotti, & Perez, 2012; Rafiqi et al., 2015; Rudmann, McConkie, & Zheng, 2003; Zagermann, Pfeil, & Reiterer, 2016). However, it should be considered that in addition to cognitive processes, pupils diameter size can be changed based on the brightness of the environment. Pupils’ diameter size increases when the environment becomes darker to obtain more light and decreases when the environment becomes brighter. Therefore, controlling the brightness of the environment and display brightness are critical factors that can affect pupils’ size and affect the reliability of the usability test results. In order to get more accurate results when testing a user interface based on pupils’ size, the calibration value for the display brightness should be subtracted from the measured pupils’ size (Liu, Zheng, & Zhou, 2019). See Table 14 for relevant studies that used pupil dilation measurement methods.

The reviewed studies demonstrate that pupil-dilation measures are effective indicators of cognitive load and usability. Pupil metrics were frequently analysed alongside other measures such as fixation behaviour, task-performance outcomes, usability questionnaires, and NASA-TLX ratings (Dobhan, Wüllerich & Röhner, 2022; Jiang et al., 2018; Majooni, Akhavan & Offenhuber, 2018). Across these investigations, greater task difficulty and reduced interface usability were consistently associated with increased pupil dilation, longer fixation durations, and higher self-reported workload scores. This alignment between physiological, behavioural, and subjective indicators provides applied evidence of the convergent validity of pupil-dilation measures for assessing cognitive load in usability evaluation, even though the reviewed studies focused primarily on applied interface assessment rather than formal psychometric validation.

Another physiological (involuntary) eye movement measurement is rate and latency of blinking which can show the attention of users. A high blink latency and a low blink rate are an indication of a high mental load (Chen et al., 2011). When users need to process the usage of an interface element that is not easily comprehensible for them and they need to devote more attention to a task, their blink rate will decrease. Therefore, the blink rate is an indication of increasing cognitive load in the area of human computer interaction (Joseph & Murugesh, 2020). See Table 15 for relevant studies that used the blink rate.

The reviewed studies indicate that blink-rate measures are effective indicators of cognitive load and usability. Blink-rate analysis was frequently conducted alongside other physiological and subjective measures such as fixation behaviour (Fowler, Nesbitt & Canossa, 2019; Li et al., 2024; Nagy et al., 2023; Sevcenko et al., 2023), pupil dilation and saccades (Nagy et al., 2023; Sevcenko et al., 2023), NASA-TLX ratings (Li et al., 2024; Mazur et al., 2019; Nagy et al., 2023), and performance outcomes such as task duration and test-score accuracy (Mazur et al., 2019). Across these investigations, higher cognitive load and reduced usability were consistently associated with decreased blink frequency and increased fixation duration, reflecting sustained visual attention and mental effort. The convergence between blink-rate patterns and physiological, behavioural, and self-reported indicators provides applied evidence of the convergent validity of blink-rate measures for assessing cognitive load in usability studies, even though most investigations were designed for applied interface evaluation rather than formal psychometric validation.

When measuring cognitive load using eye movement measurement methods, there are three factors including individual interaction, social interaction, and environmental factors that should be taken into account (Jetter, Reiterer, & Geyer, 2014).

Individual interaction describes the way users interact with a system interface including menus, content, and tools. Based on the system design and the activity that users perform with the system, they will be required to focus on a specific part of the interface which can influence users’ fixations. Thus, interface layout and the visual presentation of content can also influence users’ fixation location, duration, and rate without changing their cognitive load. Also, the interface structure as well as the task at hand can influence saccadic eye movements including length, velocity, and angle of saccades (Zheng, 2017). For example, while users perform a visual analytic task in spreadsheet applications, they need to switch between different tools, worksheets, and charts which can affect fixation and saccadic eye movements. Another example is the difference between eye movements when users are searching for information by reading a text versus checking an image.

The other factor that can influence saccadic eye movements and fixations without affecting users’ cognitive load is the input and output technologies including mouse, keyboard, and touch screen displays that are used to interact with the interface. Input devices might require users to fixate on the input in addition to the visual information on a display (Gao & Sun, 2015). For example, graphic designers often need to switch between desktop displays, and digital pen and tablet to draw an artwork (Keim et al., 2008). Input and output devices can also affect pupil dilation and blinking behaviour as the luminance of the devices such as mobile phones, tablets, and laptops can be changed frequently based on the brightness of the environment (Pfleging et al., 2016). Thus, in terms of individual interaction, it is important to consider the software content, the task at hand, and the input and output devices as additional elements that can influence eye movements without changing cognitive load.

Social interaction describes the social aspects that influence the way we collaborate with the use of the system. Some systems such as online games, telecommunications applications, and e-learning platforms are inherently social, as users need to work with the system in groups and have communication and perform coordination activities with the other users. Depending on the activity, the number of users that interact with each other, and social roles of users, fixations and saccadic eye movements can be affected without changing cognitive load (Zagermann, Pfeil, & Reiterer, 2016). Switching focus between different users and interactive devices is exhausting for the eyes and consequently it can increase blinking rates and the velocity of eye movement as well as fixations and saccades. Therefore, the interference of eye movements through communication with the other users should be considered in order to increase the reliability of the usability test results when they are measured using eye movements (Zagermann, Pfeil, & Reiterer, 2016).

Environmental factors describe the characteristics of the physical environment such as the office, lab, class, or conference room where users interact with the system. Since each environment has its own characteristics such as temperature, humidity, and luminance with different types of furniture, equipment, and devices, users’ attention and consequently eye movements can be changed during the usability test without changing cognitive load. Thus, during a usability test the fixations should be evaluated with respect to the physical surroundings. Also, the environmental characteristics can exhaust the eye and increase blinking. For example, the environmental luminance has the highest impact on pupil size. Therefore, environmental factors should be taken into account when conducting a usability test using an eye movement measurement method as it can interfere with eye tracking measurements in several different ways (Chen et al., 2016).

Electrodermal Activity (EDA)

EDA is one of the characteristics of the human body that causes continuous changes in the electrical properties of the skin (Boucsein, 2012; Critchley, 2002). EDA theory indicates that skin resistance varies based on the state of sweat glands in the skin and the sympathetic nervous system is responsible in controlling sweating (Martini & Bartholomew, 2001). By arousing the sympathetic nervous system, the activity of sweat glands is increased, enhancing skin conductance. Therefore, skin conductance can be measured as an indication of physiological arousal (Carlson, 2013). In other words, based on environmental factors and a person’s cognitive state, emotional arousal changes can increase sweat gland activity and consequently lead to an increase in skin conductance. It should be noted that the EDA signal is representative of the intensity of an emotion but not emotion type.

In applied usability testing, EDA is measured by attaching very thin sensors around the fingers or wrist with the sensors sending the signals wirelessly to a computer, then the computer quantifies moments of significant increased arousal (iMotions, 2021). For example, a spike in conductance while a user interacts with a complex or confusing interface element may indicate an increase in cognitive or emotional load at that moment. This allows researchers to align EDA data with specific user interactions, helping identify which interface components contribute to elevated cognitive demand.

Due to the simplicity of the EDA measurement method, its accuracy in detecting changes in human mental states, and the straightforward procedures for quantifying EDA data, it is a suitable technique for measuring cognitive load during usability evaluation (Georges et al., 2017; Nourbakhsh et al., 2017; Veilleux et al., 2020). There are two other commonly used terminologies for EDA, namely, galvanic skin response (GSR) and skin conductance response (SCR) that fall under the umbrella term of EDA, and they all essentially refer to the same concept. In fact, EDA has been known as GSR and SCR historically. In the recent studies, researchers mostly used the acronym EDA, however, in older studies either GSR or SCR were used (Boucsein, 2012). See Table 16 for relevant studies that used EDA.

The reviewed studies demonstrate that EDA measures are effective indicators of cognitive load and usability. EDA was employed either independently or alongside other indicators such as interviews and performance measures (Barathi et al., 2020) or subjective questionnaires assessing perceived task difficulty (Mandryk, Atkins & Inkpen, 2006). Other investigations applied EDA alone to capture physiological arousal associated with increased mental effort (Nakasone, Prendinger & Ishizuka, 2005; Shi et al., 2007). Across these studies, higher task complexity or engagement consistently produced elevated skin-conductance responses, reflecting increased cognitive and emotional activation. The convergence between EDA trends and subjective or behavioural indicators provides applied evidence of the convergent validity of EDA as a physiological measure of cognitive load in usability contexts, even though most studies were designed for applied interface evaluation rather than formal psychometric validation.

Heart Rate Variability (HRV) and Blood Pressure

HRV is a measure of variation in the time interval between heartbeats, and it is an indication of changes in blood pressure and mental stress that are controlled by the autonomic nervous system (ANS) and hypothalamus in the brain (Buccelletti et al., 2009; Solhjoo et al., 2019). By increasing cognitive load, both sympathetic and parasympathetic components of the ANS increase, that impact the HRV (Goldstein et al., 2011; Solhjoo et al., 2019). The relationship between HRV and cognitive load is indirect: as cognitive load increases, heart rate, and blood pressure tend to rise, which in turn leads to a reduction in HRV (Ayres et al., 2021; Hjortskov et al., 2004). It should be noted that HRV is a more accurate and sensitive measure of cognitive load compared to heart rate or blood pressure alone, since HRV reflects the central pathway in the cardiovascular mechanism; however, the blood pressure is more influenced by the working muscle conditions (Hjortskov et al., 2004).

HRV is more successful with short-duration basic tasks such as binary decision tasks and to measure large differences in cognitive load such as differences between mentally active and mentally inactive periods (Paas, van Merriënboer & Adam, 1994). Based on several studies, using HRV in longer-lasting learning tasks can decrease the validity and sensitivity of the results (Ayres et al., 2021). Therefore, HRV is more appropriate to evaluate the location of a specific component of the interface that only has one step rather than evaluating a long task with the use of the interface.

HRV can be measured with the use of electrocardiogram (ECG), blood pressure, ballistocardiogram, and photoplethysmography (Bruser et al., 2011; Brüser et al., 2012). In order to evaluate cognitive load, ECG is the most common approach of measuring HRV carried out by placing electrodes on the chest skin (Forte, Favieri & Casagrande, 2019).

Most of the studies that evaluated the efficiency of the HRV method in measuring cognitive load in the context of usability testing, used HRV with a combination of other approaches such as EDA and EEG. See Table 17 for relevant studies that used HRV alone or in conjunction with other measurement methods.

The reviewed studies demonstrate that HRV measures, particularly when combined with other physiological indicators, are effective tools for assessing cognitive load and usability. HRV was consistently evaluated alongside EDA (Chan et al., 2020; Collins et al., 2019; Rajavenkatanarayanan et al., 2020; Solovey et al., 2014), EEG (Gupta et al., 2019; Zhou et al., 2020), and subjective feedback (Chan et al., 2020). Across these investigations, increased task difficulty and higher cognitive demands were associated with reduced HRV and elevated EDA or EEG activity, reflecting heightened physiological arousal and mental effort. The convergence across multiple physiological channels provides applied evidence of the convergent validity of HRV as a complementary indicator of cognitive load in usability evaluation.

The reason HRV has not been used alone as a measure of cognitive load is that HRV alone may not provide a complete picture of cognitive load. While changes in HRV have been linked to cognitive load, they can also be influenced by other factors, such as physical activity, stress, or emotions. Moreover, different types of cognitive tasks may elicit different patterns of HRV, making it difficult to interpret HRV data without additional measures. Therefore, HRV is often used in conjunction with other cognitive load measures to provide a more comprehensive assessment of cognitive load (Bong et al., 2016).

Facial Expressions

Measuring facial expression could identify the mental load based on micro-movements of facial muscles, such as frowning, which are an indication of the emotional state of an individual (Cacioppo et al., 1986). Both positive and negative emotions lead to different changes in the face muscles such as changing the form of mouth, eyebrows, and cheeks. In order to measure facial expression, the facial movements are recorded with a regular high-quality camera and the start, duration, and end of each facial movement are taken into account (Freitas-Magalhães, 2013). Commonly an angry face indicates high mental load, but neutral and happy faces indicate low mental load (Johanssen, Bernius & Bruegge, 2019; Pecchinenda & Petrucci, 2016). These interpretations are typically made within the same participant, by comparing moments of higher versus lower arousal during task performance, rather than between different participants.

One of the issues in using facial expressions compared with the other measurement methods of cognitive load such as physiological and task performance is that the procedure will show less variation from high to low arousal when there is no affective interference (Hussain, Calvo & Chen, 2014). Therefore, it cannot be used to compare different variations of cognitive load during usability testing when there is not a significant difference between the mental loads. Although, there are different studies that have used facial expression in order to evaluate the usability of different interfaces, there are not many studies in the area of evaluating facial expression in measuring cognitive load for usability test purposes. See Table 18 for relevant studies that used facial expression alone or in conjunction with other measurement methods.

The reviewed studies demonstrate that facial-expression analysis, whether used independently or in combination with other cognitive load measures, provides an effective method for assessing usability and cognitive load. Facial-expression data were frequently analysed alongside physiological indicators such as EDA and HRV (Dargent et al., 2019; Thüring & Mahlke, 2007), performance outcomes (Machado et al., 2018; Thüring & Mahlke, 2007; Xu et al., 2018), self-reported usability or satisfaction questionnaires (Thüring & Mahlke, 2007; Xu et al., 2018), and eye-movement measures including pupil dilation or blink rate (Corcoran et al., 2012; Dargent et al., 2019). Across these investigations, increased cognitive load and reduced usability were consistently associated with negative or strained facial expressions (e.g., frowning and brow tension), whereas positive affective expressions corresponded to lower mental effort. The convergence of facial-expression patterns with behavioural, subjective, and physiological indicators provides applied evidence of the convergent validity of facial-expression analysis for detecting user workload and emotional engagement during usability evaluation, even though most studies were designed for applied assessment rather than formal psychometric validation.