1. Introduction

In recent decades, papers on human-computer interaction (HCI) have discussed effects in usability (UX effects) that are conditioned by factors outside the process of task execution (sometimes referred to as contextual factors). However, as a rule, UX research is focused on identifying the influence of one particular contextual factor ‘all other things being equal’; combinatorial effects formed by the simultaneous impact of heterogeneous contextual factors are hardly studied so far, although such impact is almost always present and inevitable.

A generalized model of the combination of such effects, including four groups of factors (interface properties, the nature of the task, user traits, and experiment conditions), was proposed by Sauer and Sonderegger in 2011 as the ‘contextual fidelity model’ (CFM). According to this model, the assessment of efficiency of communication between an interactive system and a user should be made taking into account all external and internal conditions of such interaction. ‘Universal’, context-independent understanding of ‘interaction experience’ is, thus, deeply incorrect, as test results are influenced by many factors related to both the subject of evaluation (the current psychophysiological state of a user and his/her reactions to the properties of the tested product and tasks) and to the objective circumstances of the test procedure (timing, physical surroundings, and social environment of the process). This approach to usability assessment studies the combinatorial (including, e.g., cumulative or compensatory) effects of the above factors cast upon user’s cognitive and emotional processes during interaction with the interface [1-4]. Understanding of these psychophysiological effects could significantly advance the theory and methodology of efficient interface design for information interactive systems and data visualization technologies.

The ‘contextual fidelity model’ has already been applied to university interfaces and experimentally created web portals. However, it has not yet been used to evaluate portals of large-scale media projects, neither in Russia nor abroad, while the importance of user interaction with media portals has grown critically in the recent decades in terms of both time and user engagement. Moreover, news portals are design objects of increased complexity in terms of aesthetics, content filling, speed of content change, page diversity, etc. This paper proposes to partially fill this gap, as it examines the combinatorial effects of the CFM factors for web portals of news agencies in Russia and China.

2. Problem statement

The proposed study examines the combinatorial effects on user experience of four CFM factors, namely the product (website) design quality at two different levels of page organization, task properties (‘quick’ and ‘slow’ task), and cultural affiliation. For this study, this means measuring the impact of graphical interfaces from different countries with different design quality at two different levels of page layout (micro- and macro-) under two types of task conditions – at speed (the ‘fast’ task) and without time limit (the ‘slow’ task) – upon the users’ psychophysiological states after task performance. We employ metrics of users’ psychophysiological states – cognitive efficiency (‘productivity’) and the level of psycho-emotional strain (stress and fatigue levels) as dependent variables. Thus, we measure the complex cognitive-psycho-emotional dysfunctionality of the user that arises depending on the combination of groups of factors that need to be taken into account.

3. The theory of human-computer interaction and its extensions

3.1. Independent variables: Factors of ‘contextual fidelity model’ and graphical structure of the interface

To date, methods of studying interface efficiency have relied on four independent approaches, each of which discusses one of the factor groups of the ‘contextual fidelity model’ as the leading one.

The contextual approach discusses the effects generated by the immediate environment and conditions of the assessor’s activity during the experiment. In particular, this line of studies investigates the dependence of interaction quality on the presence of other people [5] or the overall HCI format [6]. Such papers put attention to users’ reactions to stimuli that stem from the external environment and can affect the intensity of cognitive processes, such as the presence of other people, intragroup interaction, interpersonal communication between the mentor and the assessor, and the proximity of the test conditions to the work context. As a rule, physical and social conditions of the experiment are discussed separately; however, recently, interest has risen towards analyzing their joint impact upon users.

The task properties approach discusses the dependence of user experiences on the structural [7] and cognitive [8] complexity of the tasks being performed.

The impact of demographic and psychographic user traits upon the results of interaction is the focus of interest of the third research area that measures the impact of user properties upon usability [9-11].

However, so far, the most intensely developed area is the one focused upon the dependence of user experience upon product design. The focus here, in particular, is the linkage between aesthetic quality of web pages, on one hand, and perceived usability [12], subjective user satisfaction, user performance, and design functionality [13], on the other.

The focus of the latter studies is, as a rule, the relationship between objective indicators of user productivity and subjective factors of visual perception that determine the evaluation of interface quality. At the same time, such studies are characterized by contradictory results in the study of these problems using empirical methods. Thus, if some of them demonstrate a linear dependence of user productivity on the aesthetic quality of design, others prove the negative influence of interfaces of high aesthetic quality on task solving. In general, the studies of product properties are still characterized by a reliance on rather superficial and subjective scholarly approaches. In addition to a conditional understanding of the context of product use (the conditions of which are often too close to laboratory conditions in their sterility), an obvious shortcoming of most studies is a too general, dimensionality-reducing approach to the evaluation of design quality based on subjective indicators (such as, e.g., aesthetic evaluation). Meanwhile, understanding design only as a syncretic, undifferentiated figurative whole is also wrong; it may correspond to some known factors of aesthetic judgment, but, at the same time, contradicts the fundamental laws of visual perception as successive genesis of image. In other words, the integral image itself is formed first of all as a result of a multitude of subjective reactions to individual components of the perceived stimulus, and is assembled of them. This understanding of the inherently discrete nature of user experience corresponds to some studies in the field of human-computer interaction [14], according to which various levels of layout affect differently the processes of visual perception and intellectual activity of the assessor.

In this regard, it is particularly important to differentiate two structural levels in page layout, namely the micro- and macro-levels [15], each of which is associated with different cognitive-perceptual modes. This taken into account, the formation of user experience becomes a multidimensional process in which a user reacts to many discrete elements of design, such as, e.g., page layout, correspondence between graphics and text, font style, line spacing, and character height. Theoretically, the differentiated impact of the layout upon the user’s cognitive processes is supported by the theory developed by Velichkovsky [16]. It describes human cognitive activity as a system of two mutually exclusive modalities, namely perception in the mode of concentration on the object (the ‘focal’ mode) and the process of orientation within the conditions of arbitrary choice of stimuli (the ‘ambient’ mode). Both modes contribute to information processing, but do not overlap: Thus, in the focal mode, the user is focused on recognizing local details with a reduced field of view, while, in the ambient mode, the field of view is significantly expanded and attention is dispersed in the search for orientational stimuli. Looking at the layout structure through the prism of cognitive modes provides for an accurate analytical perspective for the studies of its functionality, as differences between the macro- and micro-levels may directly affect the users’ psychological states, and, as a consequence, determine the overall efficiency of the interaction with a given website. Located at different levels of the compositional architecture, nevertheless, the elements of design constitute local systemic unities interlinked via their common impact upon the cognitive processes:

• the macro-level of composition (the F-pattern, color zoning, modular layouting, and content creolization) combines the elements that organize the overall architectonics of a web page and control the efficiency of the cognitive search modes described above;

• the micro-level of composition (adaptability, font size and contrast, type of typeface, spacing and line length) combines criteria that provide detailed study of content and control the effectiveness of the focal (successive) cognitive mode.

Analyzing the impact of each of the layout levels (within CFM) upon user experience is of undoubted value for understanding the plastic, non-linear dynamics of psychophysiological processes that constitute the very essence of human interaction with an interactive object. The impact of product design upon the user via a graphical interface generates a complex user reaction, in which cognitive, sensory-hedonistic, and emotional experiences are intertwined, often working to reinforce or suppress each other. Therefore, without a detailed study of such dynamics, it is not possible to further improve the methods of interface design and its quality assessment. Therefore, careful analysis of the differential impact of layout elements upon user experience is of undoubted importance for UX research, as it provides for precision and depth so necessary at the current stage of HCI studies.

3.2. Dependent variables: Cognitive and emotional-affective modes of user experience as part of a user’s complex (dis)functionality

The realization of such research is, however, impossible without defining dependent variables clearly affected by the layout features; these should be capable of capturing changes in cognitive and affective processes of user experience.

What should these dependent variables be?

In accordance with the objectives of most empirical HCI studies, such variables should have properties that satisfy at least three validity criteria:

• sensitivity – the variables should have high sensitivity to changes in testing factors;

• practical orientation – the variables should connect with real users’ activities and manifest themselves when solving the majority of real-world tasks;

• universality – they need to be independent from particular test factors, i.e. retain diagnostic relevance for different tasks, products, and test situations.

The methodological value of such process indicators lies in the fact that their combinations with each other can create a recognizable pattern that would allow for identifying the actual state of a user quite accurately. As a number of studies [14, 17-19] have shown, among many parameters of psychophysiological activity, valid results of the evaluation of user reactions are ensured by four of them: (1) The amount of working memory and (2) mobility of neural processes for the cognitive sphere, as well as the level of (3) stress and (4) fatigue for the emotional-affective sphere.

Thinking and memory are the basic abilities of the users’ cognitive sphere; that is why the quality of user experience is overwhelmingly evaluated via them. Activation of intellectual resources in the process of task solving implies the manifestation of a whole range of abilities, including flexibility of thinking, its adaptability to changing conditions, and capacity of switching between tasks. The main focus of measurement in this case is the mobility of nervous processes, a parameter that characterizes both the strength and lability of the nervous system. Being a generalized indicator of autonomic support of cognitive activity, this parameter can act as a key indicator of mental efficiency per time unit (we will call it ‘productivity’ further on).

The second most important focus for measuring cognitive efficiency is working memory, the resource that provides for and ensures thinking. The diagnostic value of working memory lies in its high sensitivity to major factors of the ‘contextual fidelity model’, that is, to the users’ emotional experience, cognitive complexity of tasks, and social context.

For the emotional-affective sphere, the most representative foci are the assessment of stress and fatigue levels. Their high sensitivity to the intensity of stimuli was noticed long ago [20-22]. It is also known that the development of these experiences is least susceptible to volitional control, which makes them convenient for operational monitoring of emerging user dysfunctions.

The differentiating features relevant to this study that constitute the complexes of symptoms of stress and fatigue in the three main modalities of human psychophysiological activity, have been earlier systematized by us based on the relevant academic studies [17-19] and are presented in TABLE 1.

TABLE 1: Signs of loss of productivity and increase in psycho-emotional states of stress and fatigue

4. Methodology of experiments

4.1. Method

In the course of the experiment, we investigated the impact of the interface of four websites upon an audience of Russian-speaking students. In assessing the changes in user experience, we focused on two types of processes, namely cognitive (‘productivity’) and emotional (‘stress’ and ‘fatigue’). To help evoke these states, we designed two types of tasks, namely a task of accelerated content search (the ‘fast’ task) and a task with no time limit (the ‘slow’ task).

To measure the quality of user experience, we chose the following indicators:

4.1.1. Productivity evaluation

This parameter was measured using two psycho-diagnostic assessment methods:

• ‘Numbers Arrangement’ test for the mobility of nervous processes [23: 552-553];

• a test of working memory capacity.

4.1.2.Assessment of psycho-emotional state

This parameter was measured using two methods:

• Spielberger-Hanin test for situational anxiety (URL: https://psytests.org/result?v=sphA3oc);

• Fatigue Assessment Scale (FAS) test for the degree of fatigue (URL: https://www.waso 75).

All psycho-diagnostic methods were applied before and after the experiment. The difference between the indicators provided for clear evidence of the influence of certain interface design factors on the users’ state.

The tasks were developed using the design of web portals of large-scale media outlets in Russia (RT and RIA Novosti news agencies) and China (CGTN and Xinhua news agency) with interactive elements and varying quality of micro- and macro-levels of webpage composition. The layout quality was measured using the method for calculating the usability index for each of the two layout levels (U-index) elaborated and tested previously by our team [14]. In accordance with this methodology, websites with differences in the design at one of the composition levels and identical design solutions on the other were compared (see TABLE 2):

• : CGTN vs. RT;

• : Xinhua vs. RIA Novosti.

TABLE 2: U-index scores for the four portals

4.2. Research design

The tests were performed in eight groups of five assessors each; four groups worked with the ‘fast’ task and the other four with the ‘slow’ task. We have developed the structure of the experiment that consisted of the following stages:

1. Pre-start testing of the productivity and the psycho-emotional state parameters.

2. Performing tasks at a certain speed on CGTN vs. RT websites; RIA Novosti vs. Xinhua.

3. Final testing of performance and psycho-emotional state parameters to determine changes.

Then, we calculated the differences (Δ) in productivity levels (in particular, in mobility of neural processes and working memory), as well as in the levels of stress and fatigue.

5. Experimental results and their discussion

The experimental results are presented in TABLES 3 and 4.

TABLE 3. Results of testing sites with differences at the macro level of the layout

TABLE 4. Results of testing sites with differences at the micro-level of the layout

1.      The following observed effects are characteristic of the situation (CGTN vs. RT; TABLE 3).

First, in general, the macro level of design significantly affects performance growth and, hence, cognitive intensity: In the case of high U-index scores (CGTN; U=6), both fast and slow tasks are characterized by an increase in mobility of nervous processes.

In particular, at the CGTN site, we can observe changes in the mobility of neural processes when performing tasks of both types: For the ‘fast’ task, the growth in test scores of neural mobility and working memory efficiency is Δ = 1 and Δ = 5, respectively, which can be qualified as weak and moderate growth, respectively. At the same time, for the ‘slow’ task, the cumulative growth of both cognitive parameters is more significant, namely Δ = 4 and Δ = 11. Probably, the difference in the growth rates of cognitive intensity in this case relates to the insufficient complexity of the test task – thus, at the starting stage in the ‘fast’ mode, the index of users’ cognitive efficiency was already quite high (9 points against the starting 6 points for the ‘slow’ task). Under the conditions of initially high level of nervous system readiness, assessors simply did not need much time and effort to solve it. The growth of performance in this case rather indicates the preservation of a high concentration mode when performing tasks at a fast pace in a short period of time.

Second, the difference in the macro-level design of the web page turns out to be significant for the formation of negative emotional states of the user. Both in the ‘fast’ and ‘slow’ tasks, the same effects are observed: In the case of high U-index on the macro-level (CGTN), the level of stress and fatigue remains unchanged, which characterizes the ability of users affected by the pages with such high U-index to maintain a stable mental state and effectively cope with stress while performing tasks at different speeds. In the case of a lower U-index (U=4, for RT), an increase in both negative emotional states is observed. This result suggests that although ‘slow’ tasks give users more time to think and process information, prolonged cognitive load still leads to fatigue accumulation. This may be due to the stereotypical nature of cognitive operations.

It is noteworthy that, although the level of fatigue increased differently when performing ‘fast’ and ‘slow’ tasks, in general, the speed of task performance did not significantly affect the level of fatigue. This finding refutes the traditional view that ‘fast’, forced tasks cause greater fatigue. It is more likely that other factors, such as task complexity, the initial state of the user, and the length of the interaction experience, are behind the task type effect on fatigue levels.

However, there is a distinct combinatorial relationship between the macro-level design and task speed. Thus, in the case of a higher U-index (CGTN), the change in mobility of nervous processes is less sound than that for a ‘slow’ task. In other words, the growth in cognitive performance slows down as the speed of the task increases. This probably is a sign in favor of a direct correlation between the period of adaptation to cognitive load and the degree of mobility of nervous reactions: The slow task provides for more opportunities for cognitive acceleration. A similar trend is observed with the amount of working memory: As speed increases for the task performed on a website with a better macro-level design, the amount of working memory decreases. The ‘slow’ task better activates working memory resources, while ‘fast’ tasks redistribute cognitive resources in favor of speed of information processing, to the detriment of working memory capacity.

2.      For the situation (Xinhua vs. RIA Novosti, TABLE 4), differences at the micro-level of design also turn out to be significant for the formation of performance dysfunctions and have the following features.

First, for productivity, the design with lower U-index (Xinhua, U=4) turns out to be sensitive to the speed of the task. The higher the speed, the more productivity decreases.

Second, for the emotional state, there is a cross-cutting effect for both websites. As the speed increases, the stress level increases, too. This can be explained in the following way: Urgency makes the user feel the complexity of the task more acutely. In turn, in the case of a ‘slow’ task, the growth of stress is not detected.

Cultural differences showed multidirectional dynamics, which may indicate the superiority of a specific portal design over cultural patterns of web design and its efficiency. Thus, in the case of differences at the macro level, the CGTN portal shows much higher efficiency in general than the RT portal, as it allows the users to increase productivity on both the ‘fast’ and ‘slow’ tasks, while RT significantly reduces user productivity and increases both stress and fatigue. But in the situation of differences at the micro level, the situation is the opposite: the Chinese news agency Xinhua shows low usability efficiency of the portal, while RIA Novosti allows the assessors to maintain the cognitive and emotional focus nearly unchanged.

6. Discussion and conclusion

Thus, the experimental data reveal two different patterns of the joint influence of a website’s compositional architecture and the content of the tasks being performed.

First, in the presence of macro-level differences in the website’s layout and the absence of differences at the micro level, the speed of users’ navigation on the website significantly affects their cognitive efficiency. At the same time, the users’ emotional states exhibit relative stability, which makes us assume that, for this level of page layout, fluctuations in cognitive efficiency, rather than in user emotional states, are closely related to the content of the tasks and design features.

Second, for differences at the micro level, the speed of navigation on a website begins to affect the negative psycho-emotional state of users. According to the results of the study, the user stress index increased both on the Xinhua website and on the RIA Novosti website. Moreover, for Xinhua, a significant decrease in the mobility of users’ nervous processes was observed regardless of the speed of completing the task. This result suggests that differences at the micro level of design can cause psychological stress in users, which, in turn, determines the speed of their psychophysiological reactions.

The role of cultural differences requires further study within the ‘contextual fidelity model’, since both here and in our earlier studies this factor appeared to be unstable or non-decisive [24]. However, systemic differences may emerge when assessor groups work in different languages.

Acknowledgements

The study was carried out within the framework of the project ‘Center for International Media Studies’ of St. Petersburg State University, Russia, stage 4, project code GZ_F_2025 – 1, ID: 116958888.

References

1. Gilal N. G., Zhang J., Gilal F. G. The four-factor model of product design: scale development and validation // Journal of Product & Brand Management, Vol. 27, ¹6, 2018. P. 684–700.

2. Sauer J., Sonderegger A. The influence of prototype fidelity and aesthetics of design in usability tests: effects on user behaviour, subjective evaluation and emotion // Applied Ergonomics, Vol. 40, 2009. P. 670–677.

3. Sonderegger A., Sauer J. The influence of design aesthetics in usability testing: effects on user performance and perceived usability // Applied Ergonomics, Vol. 41, 2010. P. 403–410.

4. Yakunin A. V., Bodrunova S. S. Cumulative Distortions in Usability Testing: Combined Impact of Web Design, Experiment Conditions, and Type of Task and Upon User States During Internet Use // Design, User Experience, and Usability: UX Research, Design, and Assessment – 11th International Conference, DUXU 2022, Held as Part of the 24th HCI International Conference, HCII 2022, Proceedings. Cham: Springer Nature, 2022. P. 526–535.

5. Als B. S., Jensen J. J., Skov M. B. Exploring Verbalization and Collaboration of Constructive Interaction with Children // Proceedings of the IFIP 9th International Conference on Human-Computer Interaction – INTERACT 2005. P. 443–456.

6. Razak F. H. A., Hafit H., Sedi N., Zubaidi N.A., Haron H. Usability testing with children: Laboratory vs field studies // Proceedings of the 2010 International Conference on User Science and Engineering (i-USEr). P. 104–109). IEEE.

7. Sauer J., Sonderegger A. Visual aesthetics and user experience: A multiple-session experiment // International Journal of Human-Computer Studies, Vol. 165, 2022, 102837 (doi:10.1016/j.ijhcs.2022.102837).

8. Bara I., Binney R. J., Ward R., Ramsey R. A generalised semantic cognition account of aesthetic experience // Neuropsychologia, Vol. 173, ¹13, 2022 (www.sciencedirect.com/science/article/pii/S0028393222001476).

9. Aledaily A., Gannouni S., Belwafi K., Aboalsamh H. A Framework for Usability Testing using EEG Signals with Emotion Recognition // Proceedings of the 5th International Conference on Intelligent Human Systems Integration (IHSI’2022): Integrating People and Intelligent Systems, Venice, Italy. P. 22–24.

10. Gannouni S., Aledaily A., Belwafi K., Aboalsamh H. Emotion detection using electroencephalography signals and a zero-time windowing-based epoch estimation and relevant electrode identification // Scientific Reports, Vol. 11, ¹1, 2021, 7071 (doi:10.1038/s41598-021-86345-5).

11. Gannouni S., Aledaily A., Belwafi K., Aboalsamh H. Adaptive emotion detection using the valence-arousal-dominance model and eeg brain rhythmic activity changes in relevant brain lobes. // IEEE Access, Vol. 8, 2020, 67444–67455 (doi: 10.1109/access.2020.2986504).

12. De Angeli A., Sutcliffe A.G., Hartmann J. Interaction, usability and aesthetics: What influences users’ preferences? // Proceedings of the 6th Designing Interactive Systems conference (DIS-06), New York: ACM, 2006. P. 271–280.

13. Tuch A. N., Roth S. P., Hornbaek K., Opwis K., Bargas-Avila J. A. Is beautiful really usable? Toward understanding the relation between usability, aesthetics, and affect in HCI. // Computers in Human Behavior, Vol. 28, ¹5, 2012. P. 1596–1607.

14. Yakunin A. V., Bodrunova S. S. U-index: An eye-tracking-tested checklist on webpage aesthetics for university web spaces in Russia and the USA // Proceedings of the 6th International Conference on Design, User Experience, and Usability: Theory, Methodology, and Management (DUXU 2017), Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9–14, 2017, Part I, #6. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 10288. Cham: Springer International Publishing, 2017. P. 219–233.

15. Yakunin A. V. Website layout as a factor of its aesthetic appeal // Problems, prospects and directions of innovative development of science: Collection of articles from the international scientific and practical conference (October 1, 2016, Ufa) / in 2 parts. Part 2. Ufa: Aeterna, 2016. Pp. 226–231.

16. Velichkovsky B. M. Research of cognitive functions and modern technologies // Bulletin of the Russian Academy of Sciences, No. 80, 2010. P. 440–446.

17. Leonova A. B. Psychodiagnostics of functional states of a person. M: Publishing house of Moscow University, 1984.

18. Leonova A. B. Structural-integrative approach to the analysis of human functional states // Bulletin of Moscow University. Series 14. Psychology, No. 1, 2007. P. 87–104.

19. Leonova A. B., Medvedev V. I. Functional states of a person in work activity. Moscow: Moscow University Press, 1981.

20. Krasnov V. N. Affective spectrum disorders. M: Practical Medicine, 2011.

21. Kostina L. M. Methods of anxiety diagnostics. St. Petersburg: Rech, 2002.

22. Berezina F. B. Mental and psychophysiological adaptation of man. L: Nauka, 1988.

23. Kasyanov S. Psychological tests. M: Eksmo, 2006.

24. Yakunin A. V., Bodrunova S. S. Cumulative impact of testing factors in usability tests for human-centered web design. Future Internet, 14(12), 2022, 359 (doi:10.3390/fi14120359).