Healthcare virtual reality lacks standardized tools for evaluating user experience

Researchers have raised concerns about how user experience is currently measured in digital health applications built on virtual reality. Their scoping review reveals inconsistent and fragmented approaches, with many studies using tools not designed for immersive environments, leaving gaps in both clinical reliability and cross-study comparability.

The paper, How to evaluate user experience in digital health? A scoping review of questionnaires in virtual reality applications, was published in Frontiers in Digital Health. The authors reviewed 325 records across PubMed, Web of Science, and Embase, ultimately including 17 studies that examined adults, both healthy and clinical populations, using VR in healthcare. Their findings highlight a lack of standardized tools and call for a comprehensive eight-domain framework to guide future evaluations.

How is user experience currently assessed in healthcare VR?

The review shows that most studies rely on self-reported questionnaires completed immediately after VR sessions. These instruments aim to capture usability, presence, motivation, and other subjective impressions of virtual experiences.

Popular tools include the System Usability Scale (SUS), User Experience Questionnaire (UEQ and its short version UEQ-S), the ITC-SOPI for measuring presence, the Simulator Sickness Questionnaire (SSQ), and the Intrinsic Motivation Inventory. Others, such as GUESS-18 or the Presence Questionnaire, were also reported. In addition, several studies used ad hoc questionnaires specifically designed for their interventions, though these lacked broad validation.

Healthcare VR interventions in the reviewed studies ranged widely. Participants included both healthy adults and patients recovering from stroke, those with mild cognitive impairment, Parkinson’s disease, COPD, ADHD, or lower-limb disorders. VR applications used head-mounted displays, 360-degree media, and interactive motion-sensor setups. The applications were intended for cognitive training, motor rehabilitation, wellness, or daily living skill simulations.

While questionnaires provided valuable insights, the review found that most instruments were adapted from traditional usability or motivation scales. Few were specifically designed for immersive VR healthcare, which raises questions about their accuracy in capturing the unique dimensions of these experiences.

What are the shortcomings of current UX evaluation practices?

The study uncovers multiple shortcomings in how user experience is currently measured in healthcare VR. The first issue is theoretical inconsistency. Many studies measure only isolated features of user experience, such as presence or cybersickness, without considering the broader spectrum of engagement or motivation. This fragmented approach prevents meaningful comparisons across interventions.

Second, the review finds that many tools used in VR health studies were not developed with immersive environments in mind. Instruments like the SUS or ITC-SOPI have roots in general usability and psychology research. Applying them directly to VR settings, particularly in healthcare, risks overlooking critical aspects of immersion and embodiment.

Another limitation is the heavy reliance on ad hoc scales. While tailored instruments may capture specific program features, they often lack psychometric validation, making their reliability uncertain. This weakens both the internal validity of individual studies and their contribution to building a cumulative body of evidence.

Cultural adaptability is also lacking. Few tools were validated across languages and cultural contexts, limiting their generalizability to global health applications. Moreover, nearly all studies measured user experience only immediately after a single VR session. This short-term focus neglects how perceptions might shift over repeated use, which is particularly relevant for rehabilitation and long-term wellness interventions.

What do the authors propose for the future of VR in digital health?

To address these gaps, the review identifies eight recurring domains that should guide future user experience evaluations in healthcare VR. These domains include usability and functionality, aesthetics of design, engagement, emotional state, presence, realism of environments, side effects such as cybersickness, and motivation or intention to use. Together, they provide a comprehensive structure that captures the multi-dimensional nature of VR-based healthcare.

The authors recommend that future studies design evaluations around this eight-domain framework rather than relying on fragmented or generic tools. They emphasize the need for multidisciplinary input, involving experts in human-computer interaction, clinical practice, and psychology to select or design appropriate instruments.

Longitudinal data collection is also highlighted as critical. Evaluating user experience across multiple sessions, rather than only after single exposures, would capture how patients adapt to VR interventions over time. This is essential for understanding adherence, acceptance, and eventual clinical impact.

Finally, the authors stress that tools should be validated psychometrically and culturally. Only through rigorous testing can instruments be trusted to provide reliable and generalizable data across diverse healthcare populations. By grounding evaluations in both theory and empirical validation, VR research can move toward producing results that are both comparable and clinically actionable.