⟵ back to the overview
Virtual Reality in Human Factors Research
so much more than driving simulators
by Philipp Maruhn & Lorenz Prasch
|
reading time: 10 mins
The metaverse is on everyone's lips and topics like augmented reality (AR) and virtual reality (VR) are gaining momentum—again (again? check out A Brief History of Virtual Reality). This seems the perfect time to discuss the role of virtual worlds in human factors research and what their potential looks like in the future.
Putting humans at the center of a new reality—a core vision of human factors—is now possible in even more fundamental ways than before. But let's not get ahead of ourselves. Normally, when people talk about potential, there is a graph somewhere that shows exponential growth. Of course, we are happy to oblige. We took a look at publications with the keywords AR or VR in academic journals with a focus on human factors. You will quickly notice that the application of virtual display technologies has a long tradition. And why is that so? Because they are so darn practical. They enable user-centered testing and development from the earliest stages of the design process, thus decreasing the cost of testing different alternatives. This ultimately results in faster iterations and therefore more user-centered results (which—as we can all agree on—is always awesome).
But a second observation can be derived from the data: since 2013, with the advent of a new generation of displays, there has been a virtual tidal wave of new publications. Some of these have the technology itself as the subject of their research, some just use it to achieve more meaningful results. Both are worth a look.
Especially the latter, using some form of simulator in which you can see your own body (i.e. augmented virtuality (AV), Here is our take on the definition of AR,AV,VR, etc), has a great tradition in human factors research. Three examples are driving, flight and pedestrian simulators:
Unfortunately, this is a Youtube video. While we do not use cookies or track you, Youtube does. So as soon as you hit 'play' on the video, Youtube will set it's cookies.
Driving Simulator
An example of traditional driving simulators are the ones german car manufacturer BMW is using in its facility. Dynamic, full range simulators including a highly complex and immersive simulation environment can be used both for the product development process as well as for safety assessment of new driving functionalities.
BMW Driving Simulation Centre. | #NEXTGen 2020 from BMW on YouTube.
Unfortunately, this is a Youtube video. While we do not use cookies or track you, Youtube does. So as soon as you hit 'play' on the video, Youtube will set it's cookies.
Flight Simulator
Simulators like the one one shown are used in research, education, training, and maintaining flight skills of pilots. The usually come equipped with visual displays of the virtual outer world and dynamic simulations of accellerations. Particularly useful is the high fidelity cockpit layout and therefore the lifelike haptics.
Real Human Factors in the simulator from Salos Sunesis on YouTube.
Unfortunately, this is a Youtube video. While we do not use cookies or track you, Youtube does. So as soon as you hit 'play' on the video, Youtube will set it's cookies.
Pedestrian Lab @ University of Leeds
The HIKER lab supports safe experimental research in a repeatable fashion in which a variety of variables can be varied with respect to a wide range of research questions – including Automated Vehicle design and human interaction, warning system design, and road crossing and intersection configuration.
(ITS Leeds 2021)
ITS Leeds (2021)
Highly Immersive Kinematic Experimental Research (HIKER) pedestrian lab
Link
Highly Immersive Pedestrian Simulation from Human Factors @ ITS Leeds on YouTube.
These simulations—while offering very high fidelity and good immersive experiences for research and training—are also very complex to set up and difficult to maintain.The sheer amount of space they occupy additionally makes them costly in terms of real estate as well as nigh impossible to use for distributed presentations.
This is the way
HMDs are the new kids on the block
Recently, advances in HMDs have enabled more complex and simultaneously more simple, adaptable and logistically practical setups. Instead of people having to come to a dedicated simulation facility, these headsets can—depending on the specific use-case—simply be packed up and shipped around the globe. Besides, the entire environment is often easier to adapt and thus more practical for rapid design iterations:
ARCAR: On-Road Driving in Mixed Reality by Volvo Cars
ARCAR is a proof-of-concept headset-based mixed reality experience, for use by a driver in full control of a moving vehicle. The functional prototype has been implemented using a high end video pass-through mixed reality headset, and tested on roads (closed to external traffic) at conventional driving speeds. Our implementation embodies acceptable solutions to a number of current challenges that have been in the way of in-car XR applications, and enables a wealth of research. ARCAR is intended to serve as a research platform in the near future, enabling investigations on topics which include safety, design, perceived quality, and consumer applications.
(Ghiurãu, Baytaş & Wickman 2020)
Florin-Timotei Ghiurãu, Mehmet Aydın Baytaş & Casper Wickman (2020)
ARCAR: On-Road Driving in Mixed Reality by Volvo Cars
DOI
Volvo x Varjo: Pave the Way from Varjo on Vimeo.
Analyzing pedestrian behavior in augmented reality—proof of concept
Creating virtual environments is time-consuming, especially in terms of meeting today's expectations regarding graphical level of detail and realism. Additionally, VR experiments often lack a body representation or require additional sensors to create an avatar. Due to the laboratory setting, VR simulators might fail to convey the feeling of standing on an actual street. In addition, simulators on the one hand and real-world testing on the other hand leave a methodological gap on the reality-virtuality continuum. This paper presents a novel approach for an augmented reality pedestrian simulator.
(Maruhn et al. 2020)
Philipp Maruhn, André Dietrich, Lorenz Prasch & Sonja Schneider (2020)
Analyzing Pedestrian Behavior in Augmented Reality — Proof of Concept
DOI
Studying pedestrian behavior in augmented reality from proband15 on Vimeo.
Unfortunately, this is a Youtube video. While we do not use cookies or track you, Youtube does. So as soon as you hit 'play' on the video, Youtube will set it's cookies.
Helicopter Pilot Training by VRM Switzerland
This FSTD for the H125 uses a VR headset to create an immersive 3D environment for pilots, with extraordinary renderings of the cockpit, instruments, and airframe that are true to life. The pilot sits in a small, lightweight cockpit that takes up less than 215 square feet (20 square meters) of floor space and has a total mass of just 1,279 pounds (580 kilograms). It's small enough to fit in any mid-sized aircraft hangar and can be easily transported to multiple sites. This H125 simulator also has a full-motion platform powered entirely by electricity, with remarkable responsiveness to activity within the simulation. In comparison to traditional simulator domes, which rely on screens to replicate terrain, VR goggles produce an unrestricted immersive 3D environment that responds to the pilot's head movements in real time.
(VRM Switzerland 2022)
VRM Switzerland (2022)
A brilliant partnership
Link
Airbus H125 Flight Training Device FTD Level 3 from VRM Switzerland on YouTube.
Closing thoughts
now what?
As is apparent, XR (AR, AV and VR) is a tool that has enormous potential for Human Factors Research, which is probably the reason why it has been used for quite some time now. But the setups of the past almost always had the caveat of being complex, bulky and difficult to adapt. This is the main reason why they were mostly used in a pure research context and never really made the leap into consumer-grade product development. New display technologies, namely head-mounted displays (HMDs), however, offer the opportunity to make XR more accessible. This is true not only for research purposes, but also for showcases, market research, or fast paced user-centered design cycles.
Current HMDs have built-in eye-tracking systems, which makes them particularly interesting for human factors research. In combination with stereo cameras, current HMDs can also be used as AR glasses, which significantly increases the range of application. This is why we are convinced that the use of HMDs will increase even further, gradually replacing the old technology, even though "big" simulators will always keep their raison d'être for certain research questions or training. Access to an immersive experience is now just a parcel away.
⟵ back to the overview