HCI and social science experimentation that explores or uses extended reality (XR) has been particularly impacted by the recent Covid-19 pandemic. This is due to typical deployment of XR experiments inside laboratories, and a paucity of research into how to effectively conduct remote XR experimentation. This first CHI Remote XR workshop aims to explore the current state of the art around three main themes of remote XR experimentation:
This workshop brings together researchers and practitioners in XR to explore these recently emerged themes and to imagine how effective future remote XR research might be conducted.
The workshop has three themes. Please click to see the theme in more detail
The recruitment of online participants for non-XR experiments is generally considered effective, and often happens through platforms such as Amazon Mechanical Turk. While this approach has been used for XR studies, the use of XR-specific hardware has limited the participant pool (e.g. only 1.4% of Amazon Mechanical Turk respondents have access to head-mounted VRdisplays).
Participants also may have to have access to a dedicated physical space (e.g. devices usually require an empty 2×2 metres, while AR experiments may need fixed locations), as well as other common experimental requirements (e.g. no distractions).
This limited participant pool raises two questions: is it representative of wider populations, and how do we effectively access this smaller pool of users? To answer this, we need a better idea of who these users are and how to target them. In addition, it is important to understand ethical concerns of using participants in different countries and within different cultural/social/physical environments.
Laboratory settings allow researchers to setup and capture many different types of data from a participant, which have previously not been practical for remote XR studies (e.g. physiological data, external cameras, bespoke hardware interactions). It is reasonable to suggest that remote XR experiments are not yet able to easily recreate this level of data collection.
However, modern XR-enabling hardware (such as consumer VR kits) allow for many types of data collection that were previously difficult to collect or required bespoke setups. Different variations of XR-hardware also enable further data collection (such as HMDs with in-built eye-tracking).
There are also novel approaches to understanding human activity that are possible via XR-hardware that have previously used dedicated sensors, such as using microphones to measure exercise exertion or body or head movements for focus on interest and interactional attention.
We believe it is important for the XR research community to outline both the limitations and potential of existing XR-hardware, as well as imagine what an idealised XR-hardware-as-data-collection device might look like.
Software applications for XR development have been traditionally developed with the assumption of lab-based experimentation. Work is being done to simplify the data collection step for XR experiments built in Unity.
However, there is not yet an approach that is dedicated to the requirements of remote studies. In fact, we still need to establish the requirements for the development of a software framework that allows the effective implementation of remote XR studies.
This should not only include the data collection methods, but also libraries to transfer and store the data safely and easy-to-setup environments to run studies. This type of “encapsulated experiment” could also improve replication and transparency, as theorised by Blascovich, and allow for versioning of experiments, in which researchers can build on perfect replicas of other’s experimental environments and processes.
Questions remain over what are the constant features that should be at the core of most XR remote research and that should necessarily be present and available for researches who want to deploy a remote XR user study.
Research experiments in the XR/MR/VR field has traditionally taken place in dedicated space and laboratories and been supervised by the researcher. With COVID-19, the sudden transition to remote experimentation has left many researchers without opportunities to carry on their user studies. This is partially due to the lack of remote-oriented solutions for XR research.
This workshop will bring together researchers and practitioners in XR research with the goals to:
Participants should submit position papers (max 3 pages in CHI ACM Master Article format) about their XR research and practice addressing the themes listed at the workshop website. Position paper submission is to firstname.lastname@example.org. Participants will be selected based on the quality of XR research and practice and with a view to creating a balance of topics in the workshop.
Please note that at least one author of each accepted position paper must attend the workshop and that all participants must register for both the workshop and for at least one day of the CHI 2021 conference.
The workshop is designed to provide an opportunity for researchers to discuss the needs and limitations of remote XR practice. The aim, broadly, is to collaboratively imagine practical futures for idealised remote XR research processes, and outline the requirements to reach these.The workshop will take place in an online environment as a virtual activity during the conference.
The structure of the workshop is summarised in the image above, and consists of three sessions, one for each theme outlined above: participant recruitment, XR hardware and data collection, and encapsulated experiments.
Each of the first two sessions consist of two sprints: the first for discussing relevant prevalent challenges, and the second for imagining ways to overcome these. In each sprint, participants will be split into two breakout rooms to allow for more intimate, involved discussions. At the end of a sprint, the groups will reconvene to share results.
The third session is an open discussion, partially based on learnings from the first two, and leveraging a cooperative shared annotation space for sharing and arranging ideas. The workshop will take place in a normal video conferencing application and additional platforms will be used for the interactive activities (e.g. Miro, Mural or Google slides). The total duration of the workshop, including breaks, will be 3.5 hours.
Our aim for the workshop is to collect the needs and limitations of current XR research practices and to imagine how the future research in this field could be. With the information collected during the workshop we will:
|Ensuring the Reliability of Remote VR Research Data. Melynda Hoover, Lucia Cherep, Jonathan Kelly, Stephen Gilbert presentation|
|Model-based Task Analysis and Large-scale Video-based Remote Evaluation Methods for Extended Reality Research. Yalda Ghasemi, Heejin Jeong presentation|
|Adaptive Moderated Research: Lessons Learned in Redesigning a Moderated Virtual Reality Collaboration Study. Kaitlyn Ouverson, Angelica Jasper,Stephen B. Gilbert, Nick Wilson, Peggy Wu presentation|
|Conducting a Remote Virtual Reality Experiment during COVID-19. NABIL CH and ALBERTA A. ANSAH, ATEFEH KATRAHMANI, JULIA BURMEISTER, ANDREW L. KUN, CAITLIN MILLS, ORIT SHAER, JOHN LEE presentation|
|Challenges in Running Social VR Studies. A Tale of a Covid Procedural Pivot. Victor Jialang Li, Sean Fernandes, Joshua Mcveigh-Schultz, Kathering Isbister presentation|
|Remote Collection of Physiological Data in a Virtual Reality Study. Michal, MG, Gnacek, Ellen, ES, Seiss, Theodoros, TK, Kostoulas, Emili, EB, Balaguer-Ballester, Charles, C, Nduka, presentation|
|Reflections on Remote XR Experiments During the COVID-19 Pandemic. Pejiman Sajjadi, Jiayan Zhao, Jan Oliverwallgrun, Mahda M. Bagher, Alexander Klippel presentation|
|Remote XR Studies: The Golden Future of HCI Research? Florian Mathis, Xuesong Zhang, Joseph O’Hagan, Daniel Medeiros, Pejman Saeghe, Mark Mcgill, Stephen Brewster, Mohamed Khamis, presentation|
|Remote Mobile Augmented Reality for Spatial Cognition. Yu Zhao, Soumyajit Chakraborty, Jeanine Stefanucci, Sarah H. Creem-Regehr, Bobby Bodenheimer, presentation|
|Remote Virtual Reality User Studies. Louise Bryce, Mark Sandler presentation|
|Two Paths Towards the Future of Remote Studies using Social VR. David Saffo, Sara Di Bartolomeo, Liudas Panavas, Caglar Yildirim, Cody Dunn presentation|
|Researchers’ and Participants’ Experiences on Distributed User Studies Conducted in the First Year of COVID-19 Pandemic. Anil Ufuk Batmaz, Domenick Mifsud, Anthony Steed, Wolfgang Stuerzlinger, Francisco R. Ortega presentation|
|Some Lessons Learned Running Virtual Reality Experiments Out of the Laboratory. Anthony Steed, Daniel Archer, Ben Congdon, Sebastian Friston, David Swapp, Felix J. Thiel, presentation|
This workshop is organised by:
Please contact email@example.com for questions.