Mapping UI/UX Evaluation Methods, Evaluation Objects, and Measured Aspects in Comparative Studies: A Systematic Literature Review
Abstract
— UI/UX evaluation is a crucial aspect in ensuring the quality of interaction between users and systems, particularly in the context of increasingly complex digital applications. However, the wide variety of available evaluation methods often leads to differences in results and interpretations when assessing system performance and user satisfaction. This study aims to review and compare UI/UX evaluation approaches used in comparative studies. Unlike previous review studies that mainly focused on specific application domains, user perceptions, or technological trends, this study specifically maps the relationship between UI/UX evaluation methods, evaluated objects, and user experience dimensions measured in comparative studies. The method employed is a systematic literature review (SLR) following the PRISMA guidelines. Literature was retrieved from the Scopus database, and through the selection process, 13 articles met the inclusion criteria. The analysis revealed that UI/UX evaluation approaches in comparative studies are predominantly focused on comparing interaction techniques or interaction approaches, accounting for 61.5% (8 articles) of the selected studies. This is followed by comparisons of software performance, which represent 23.1% (3 articles), and comparisons of evaluation methods, which account for 15.4% (2 articles). Evaluations generally employ a combination of performance metrics, user perception measures, and additional experiential indicators such as cognitive workload and physiological responses. The findings show that each method or system demonstrates strengths in specific evaluation dimensions, highlighting the need for a multidimensional and multi-method evaluation approach to obtain a more comprehensive understanding of UI/UX. The main contribution of this study is the development of a systematic synthesis that links evaluation methods, evaluation objects, and UI/UX indicators employed in comparative studies. The findings provide a broader understanding of current evaluation practices and may serve as a reference for researchers and practitioners in designing more appropriate and consistent UI/UX evaluation strategies.
Keywords
Full Text:
PDFReferences
J. Feng and B. Yu, “Key Factors Influencing Educational Technology Adoption in Higher Education: A Systematic Review,” PLOS Digital Health, vol. 4, no. 4, Art. no. e0000764, 2025, doi: 10.1371/journal.pdig.0000764.
L. Lemma and C. Ma, “The Role of Information Systems in Business Firms Competitiveness: Integrated Review Paper from Business Perspective,” International Research Journal of Natural Science and Technology, vol. 2, no. 4, pp. 29–42, 2020.
A. I. Stoumpos and F. Kitsios, “Digital Transformation in Healthcare: Technology Acceptance and Its Applications,” International Journal of Environmental Research and Public Health, vol. 20, no. 4, Art. no. 3407, 2023, doi: 10.3390/ijerph20043407.
B. Nurohman and A. Aziz, “Analisis UI/UX terhadap Perancangan Toko Online dengan Menggunakan Pendekatan Human Centered Design,” Jurnal Ilmiah Ilmu Komputer Fakultas Ilmu Komputer, vol. 10, no. 1, pp. 27–37, 2024.
W. T. Nakamura, I. Ahmed, D. Redmiles, E. Oliveira, D. Fernandes, E. H. T. de Oliveira, and T. Conte, “Are UX Evaluation Methods Providing the Same Big Picture?” Sensors, vol. 21, no. 10, Art. no. 3480, 2021, doi: 10.3390/s21103480.
S. Ntoa, “Usability and User Experience Evaluation in Intelligent Environments: A Review and Reappraisal,” International Journal of Human–Computer Interaction*, pp. 1–30, 2024, doi: 10.1080/10447318.2024.2394724.
L. T. De Paolis and V. De Luca, “The Effects of Touchless Interaction on Usability and Sense of Presence in a Virtual Environment,” Virtual Reality, vol. 26, no. 4, pp. 1551–1571, 2022, doi: 10.1007/s10055-022-00647-1.
P. Theodorou, K. Tsiligkos, and A. Meliones, “Multi-Sensor Data Fusion Solutions for Blind and Visually Impaired: Research and Commercial Navigation Applications for Indoor and Outdoor Spaces,” Sensors, vol. 23, no. 12, Art. no. 5411, 2023, doi: 10.3390/s23125411.
M. Sandesara, U. Bodkhe, S. Tanwar, N. Kumar, M. Alazab, and P. R. Kshirsagar, “Design and Experience of Mobile Applications: A Pilot Survey,” Mathematics, vol. 10, no. 14, Art. no. 2380, 2022, doi: 10.3390/math10142380.
P. Hu, B. Gao, K. Li, and B. Gao, “Learning Interpreting in Virtual Reality: A Scoping Review,” Interactive Learning Environments, vol. 33, no. 1, pp. 347–366, 2025, doi: 10.1080/10494820.2024.2347304.
M. J. Page et al., “The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews,” BMJ, vol. 372, Art. no. n71, 2021, doi: 10.1136/bmj.n71.
B. Kitchenham and S. Charters, Guidelines for Performing Systematic Literature Reviews in Software Engineering, EBSE Technical Report EBSE-2007-01, Keele University and Durham University, 2007.
B. Kitchenham, L. Madeyski, and D. Budgen, “SEGRESS: Software Engineering Guidelines for Reporting Secondary Studies,” IEEE Transactions on Software Engineering, vol. 49, no. 3, pp. 1273–1298, Mar. 2023, doi: 10.1109/TSE.2022.3174092.
N. Nyoni, “A Narrative Guideline to Structure the Analysis Stage of Systematic Literature Reviews,” Electronic Journal of Business Research Methods, vol. 23, no. 1, pp. 1–15, 2025.
E. Kuric, “Validation of Information Architecture: Cross-Methodological Comparison of Tree Testing Variants and Prototype User Testing,” Information and Software Technology, vol. 183, p. 107740, 2025, doi: 10.1016/j.infsof.2025.107740.
N. A. Ismail, N. A. A. Majid, N. H. A. Wahab, and F. Mohamed, “A Comparative Study of Unimodal and Multimodal Interactions for Digital TV Remote Control Mobile Application among Elderly,” International Journal of Advanced Computer Science and Applications (IJACSA), vol. 12, no. 7, pp. 157–165, 2021, doi: 10.14569/IJACSA.2021.0120718.
M. Fechter, B. Schleich, and S. Wartzack, “Comparative Evaluation of WIMP and Immersive Natural Finger Interaction: A User Study on CAD Assembly Modeling,” Virtual Reality, vol. 26, no. 1, pp. 143–158, 2022, doi: 10.1007/s10055-021-00543-0.
M. Allgaier et al., “A Comparison of Input Devices for Precise Interaction Tasks in VR-Based Surgical Planning and Training,” Computers in Biology and Medicine, vol. 145, p. 105429, 2022, doi: 10.1016/j.compbiomed.2022.105429.
M. Di Vincenzo, F. Palini, M. De Marsico, A. M. Borghi, and G. Baldassarre, “A Natural Human-Drone Embodied Interface: Empirical Comparison with a Traditional Interface,” Frontiers in Neurorobotics, vol. 16, 2022, doi: 10.3389/fnbot.2022.898859.
N. Kyaw, M. Gu, E. Croft, and A. Cosgun, “Comparing Usability of Augmented Reality and Virtual Reality for Creating Virtual Bounding Boxes of Real Objects,” Applied Sciences, 2023.
A. Kudebayeva, C. Harte, N. Bryan-Kinns, and T. Stockman, “A Comparative Study Evaluating the User Experience of Spreadsheet Programming with Sheet-Defined Functional Abstractions,” International Journal of Human–Computer Interaction, vol. 41, no. 9, pp. 5445–5461, 2024, doi: 10.1080/10447318.2024.2361209.
F. Buttussi and L. Chittaro, “Locomotion in Place in Virtual Reality: A Comparative Evaluation of Joystick, Teleport, and Leaning,” IEEE Transactions on Visualization and Computer Graphics, vol. 27, no. 1, pp. 125–136, 2021, doi: 10.1109/TVCG.2019.2928304.
J. Wang, Z. Xu, X. Wang, and J. Lu, “A Comparative Research on Usability and User Experience of User Interface Design Software,” International Journal of Advanced Computer Science and Applications (IJACSA), vol. 13, no. 8, pp. 21–29, 2022, doi: 10.14569/IJACSA.2022.0130804.
J. Wang, X. Wang, J. Lu, and Z. Xu, “Investigating the User Experience of Mind Map Software: A Comparative Study Based on Eye Tracking,” International Journal of Advanced Computer Science and Applications (IJACSA), vol. 13, no. 11, pp. 1–11, 2022, doi: 10.14569/IJACSA.2022.0131101.
A. Liapis, V. Maratou, and T. Panagiotakopoulos, “UX Evaluation of Open MOOC Platforms: A Comparative Study Between Moodle and Open edX Combining User Interaction Metrics and Wearable Biosensors,” Interactive Learning Environments, 2023, doi: 10.1080/10494820.2022.2048674.
P. Tewari, “UX Principles for Modern UI/UX Design and Their Measurement: A Framework for Digital Product Excellence,” Journal of Computer Science and Technology Studies, vol. 7, pp. 898–907, 2025.
A. Berni and Y. Borgianni, “From the Definition of User Experience to a Framework to Classify ITS Applications in Design,” in Proceedings of the 23rd International Conference on Engineering Design (ICED21), Gothenburg, Sweden, Aug. 2021, pp. 1627–1636, doi: 10.1017/pds.2021.424.
DOI: https://doi.org/10.37905/jjeee.v8i2.39319
Refbacks
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Published by:
Electrical Engineering Department
Faculty of Engineering
State University of Gorontalo
Jalan B.J.Habibie Desa Moutong Kecamatan Tilongkabila Kabupaten Bone Bolango
Telp. 0435-821175; 081340032063
Email: [email protected]/[email protected]
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.