⟵ back to the overview
            Virtual Reality in Human Factors Research
            so much more than driving simulators
            
            by Philipp Maruhn & Lorenz Prasch
            
                In human factors research, the term virtual reality (VR) is usually simply equated with simulators, such as flight or driving simulators. But taking a closer look shows that this simple notion should be abandoned. New technologies open up many new setups that enable and simplify the exploration of human-machine interaction.
                
                        
               
                 
                |
                reading time: 10 mins
            
            The metaverse is on everyone's lips and topics like augmented reality (AR) and virtual reality (VR) are gaining momentum—again (again? check out A Brief History of Virtual Reality). This seems the perfect time to discuss the role of virtual worlds in human factors research and what their potential looks like in the future.
            Putting humans at the center of a new reality—a core vision of human factors—is now possible in even more fundamental ways than before. But let's not get ahead of ourselves. Normally, when people talk about potential, there is a graph somewhere that shows exponential growth. Of course, we are happy to oblige. We took a look at publications with the keywords AR or VR in academic journals with a focus on human factors. You will quickly notice that the application of virtual display technologies has a long tradition. And why is that so? Because they are so darn practical. They enable user-centered testing and development from the earliest stages of the design process, thus decreasing the cost of testing different alternatives. This ultimately results in faster iterations and therefore more user-centered results (which—as we can all agree on—is always awesome).
            
            
                
                Amount of papers published with the keywords VR or AR in academic journals with human factors focus
            
            But a second observation can be derived from the data: since 2013, with the advent of a new generation of displays, there has been a virtual tidal wave of new publications. Some of these have the technology itself as the subject of their research, some just use it to achieve more meaningful results. Both are worth a look.
            Especially the latter, using some form of simulator in which you can see your own body (i.e. augmented virtuality (AV), Here is our take on the definition of AR,AV,VR, etc), has a great tradition in human factors research. Three examples are driving, flight and pedestrian simulators:
            
            
            
                        
                
                    
                        
                            
                            
                                Unfortunately, this is a Youtube video. While we do not use cookies or track you, Youtube does. So as soon as you hit 'play' on the video, Youtube will set it's cookies.
                                
                             
                         
                     
                    
                      
                        Driving Simulator
                      
                      
                        An example of traditional driving simulators are the ones german car manufacturer BMW is using in its facility. Dynamic, full range simulators including a highly complex and immersive simulation environment can be used both for the product development process as well as for safety assessment of new driving functionalities.
                      
                      BMW Driving Simulation Centre. | #NEXTGen 2020 from BMW on YouTube.
                    
 
                 
                
                    
                        
                            
                            
                                Unfortunately, this is a Youtube video. While we do not use cookies or track you, Youtube does. So as soon as you hit 'play' on the video, Youtube will set it's cookies.
                                
                             
                         
                     
                    
                      
                        Flight Simulator
                      
                      
                        Simulators like the one one shown are used in research, education, training, and maintaining flight skills of pilots. The usually come equipped with visual displays of the virtual outer world and dynamic simulations of accellerations. Particularly useful is the high fidelity cockpit layout and therefore the lifelike haptics.
                      
                      Real Human Factors in the simulator from Salos Sunesis on YouTube.
                     
                 
                
                
                    
                        
                            
                            
                                Unfortunately, this is a Youtube video. While we do not use cookies or track you, Youtube does. So as soon as you hit 'play' on the video, Youtube will set it's cookies.
                                
                             
                         
                     
                    
                      
                        Pedestrian Lab @ University of Leeds
                      
                      
                        The HIKER lab supports safe experimental research in a repeatable fashion in which a variety of variables can be varied with respect to a wide range of research questions – including Automated Vehicle design and human interaction, warning system design, and road crossing and intersection configuration. 
                        (ITS Leeds 2021)
                            
                                ITS Leeds (2021)
                                Highly Immersive Kinematic Experimental Research (HIKER) pedestrian lab              
                                Link
                            
                        
                      
                      Highly Immersive Pedestrian Simulation from  Human Factors @ ITS Leeds  on YouTube. 
                        
                     
                  
              
            These simulations—while offering very high fidelity and good immersive experiences for research and training—are also very complex to set up and difficult to maintain.The sheer amount of space they occupy additionally makes them costly in terms of real estate as well as nigh impossible to use for distributed presentations.
            This is the way
            HMDs are the new kids on the block
            Recently, advances in HMDs have enabled more complex and simultaneously more simple, adaptable and logistically practical setups. Instead of people having to come to a dedicated simulation facility, these headsets can—depending on the specific use-case—simply be packed up and shipped around the globe. Besides, the entire environment is often easier to adapt and thus more practical for rapid design iterations:
            
            
                        
                
                    
                    
                      
                        ARCAR: On-Road Driving in Mixed Reality by Volvo Cars
                      
                      
                        ARCAR is a proof-of-concept headset-based mixed reality experience, for use by a driver in full control of a moving vehicle. The functional prototype has been implemented using a high end video pass-through mixed reality headset, and tested on roads (closed to external traffic) at conventional driving speeds. Our implementation embodies acceptable solutions to a number of current challenges that have been in the way of in-car XR applications, and enables a wealth of research. ARCAR is intended to serve as a research platform in the near future, enabling investigations on topics which include safety, design, perceived quality, and consumer applications. 
                        (Ghiurãu, Baytaş & Wickman 2020)
                            
                                Florin-Timotei Ghiurãu, Mehmet Aydın Baytaş & Casper Wickman (2020)
                                ARCAR: On-Road Driving in Mixed Reality by Volvo Cars              
                                DOI
                            
                        
                      
                      Volvo x Varjo: Pave the Way from Varjo on Vimeo. 
                    
                      
                     
                 
                
                    
                    
                      
                        Analyzing pedestrian behavior in augmented reality—proof of concept
                      
                      
                        Creating virtual environments is time-consuming, especially in terms of meeting today's expectations regarding graphical level of detail and realism. Additionally, VR experiments often lack a body representation or require additional sensors to create an avatar. Due to the laboratory setting, VR simulators might fail to convey the feeling of standing on an actual street. In addition, simulators on the one hand and real-world testing on the other hand leave a methodological gap on the reality-virtuality continuum. This paper presents a novel approach for an augmented reality pedestrian simulator. 
                        (Maruhn et al. 2020)
                            
                                Philipp Maruhn, André Dietrich, Lorenz Prasch & Sonja Schneider (2020)
                                Analyzing Pedestrian Behavior in Augmented Reality — Proof of Concept              
                                DOI
                            
                        
                      
                      Studying pedestrian behavior in augmented reality from proband15 on Vimeo. 
                        
                     
                 
                
                    
                        
                            
                            
                                Unfortunately, this is a Youtube video. While we do not use cookies or track you, Youtube does. So as soon as you hit 'play' on the video, Youtube will set it's cookies.
                                
                             
                         
                     
                    
                      
                        Helicopter Pilot Training by VRM Switzerland
                      
                      
                        This FSTD for the H125 uses a VR headset to create an immersive 3D environment for pilots, with extraordinary renderings of the cockpit, instruments, and airframe that are true to life. The pilot sits in a small, lightweight cockpit that takes up less than 215 square feet (20 square meters) of floor space and has a total mass of just 1,279 pounds (580 kilograms). It's small enough to fit in any mid-sized aircraft hangar and can be easily transported to multiple sites. This H125 simulator also has a full-motion platform powered entirely by electricity, with remarkable responsiveness to activity within the simulation. In comparison to traditional simulator domes, which rely on screens to replicate terrain, VR goggles produce an unrestricted immersive 3D environment that responds to the pilot's head movements in real time. 
                        (VRM Switzerland 2022)
                            
                                VRM Switzerland (2022)
                                A brilliant partnership              
                                Link
                            
                        
                      
                      Airbus H125 Flight Training Device FTD Level 3 from VRM Switzerland on YouTube. 
                        
                     
                 
              
            Closing thoughts
            now what?
            As is apparent, XR (AR, AV and VR) is a tool that has enormous potential for Human Factors Research, which is probably the reason why it has been used for quite some time now. But the setups of the past almost always had the caveat of being complex, bulky and difficult to adapt. This is the main reason why they were mostly used in a pure research context and never really made the leap into consumer-grade product development. New display technologies, namely head-mounted displays (HMDs), however, offer the opportunity to make XR more accessible. This is true not only for research purposes, but also for showcases, market research, or fast paced user-centered design cycles.
            Current HMDs have built-in eye-tracking systems, which makes them particularly interesting for human factors research. In combination with stereo cameras, current HMDs can also be used as AR glasses, which significantly increases the range of application. This is why we are convinced that the use of HMDs will increase even further, gradually replacing the old technology, even though "big" simulators will always keep their raison d'être for certain research questions or training. Access to an immersive experience is now just a parcel away.
            
            ⟵ back to the overview