ESR1 – Understanding AV Predictability Using Neuroergonomics

Popular scientific abstract

In the past decade, the field of car automation faced substantial progress. Modern cars are nowadays often equipped with systems such as parking assistance or adaptive cruise control, and further advancement is expected in the near future. Hence, the role of the driver will change inevitably. It will be less about driving and more about monitoring the autonomous system. This progress will allow the driver to get involved in secondary tasks, such as watching a movie or texting with a friend. However, there will still be moments when the driver has to take-over control. This can happen in situations when the system reaches a point when it cannot operate safely anymore – the system boundary. System boundaries of highly automated vehicles might include, for example, roadworks, irregular behaviour of other traffic participants, or heavy fog affecting the accuracy of sensors. In these potentially dangerous situations, there might be only a limited time for the driver to take-over.

This raises a crucial question. Given that the driver is, for example, watching a movie instead of observing and comprehending the current traffic situation, is it conceivable to expect that his or her reaction will be prompt, but still accurate and precise?  

Our goal is to make semi-automated cars more transparent and predictable, so the driver is able to accurately cooperate with the system in situations when an intervention is necessary. We believe that the car should not act as a “black box”, but rather clearly communicate its behaviour and states to the driver. We will use neuroscientific methods to get a better understanding of what is happening in the driver’s brain during interaction with a semi-automated system. Based on our findings, we will derive recommendations for designing predictable and transparent autonomous systems of tomorrow.  

Who am I?

This image has an empty alt attribute; its file name is image-1-767x1024.pngMy name is Nikol Figalová. I come from the Czech Republic, where I studied psychology at the Palacky University Olomouc. My professional interests lie in the field of cognitive neuroscience, neuroergonomics, psychometrics, and psychology of stress and well-being.

My affiliation

My host university is the Ulm University in Germany. I work at the Department of Clinical and Health psychology.

Supervisor: Prof. Dr. Dr. Olga Pollatos, olga.pollatos [at] uni-ulm.de

Co-supervisor: Prof. Dr. Martin Baumann, martin.baumann [at] uni-ulm.de

Background

As technological possibilities in the field of car automation race forward, there is a growing need to examine their potential impact on human performance (Cohen-Lazry et al., 2017). While automation may liberate the driver from many traditional driving tasks, operations such as monitoring the intelligent system and responding to take-over requests (TORs) are added (Collet & Musicant, 2019). Drivers are also expected to be more frequently involved in secondary tasks, resulting in a reduction of situation awareness (Jamson et al., 2013). There is still a long journey towards full automation (level 5 according to SAE, 2016), in which no human intervention is necessary. Hence, issues concerning interaction between a human and highly-sophisticated, yet still imperfect technical system need to be addressed.

Norman (1990) argues that appropriately designed automation systems should continually provide feedback and interact with operators in an effective manner. Peripheral visual cues seem to be useful in semi-automated driving as they can help to increase drivers’ situation awareness, trust in the system, and to decrease the take-over time (e.g. Borojeni et al., 2016; Capalar & Olaverri-Monreal, 2018; Kunze et al., 2019; Löcken, Heuten, & Boll, 2015; Van Veen, Karjanto, & Terken, 2017; Yang et al., 2017) without demanding much attention or cognitive effort, as they require different resources than focal vision (Borojeni et al., 2016, Leibowitz et al., 1982, Wickens, 2002). Furthermore, communicating the current state of the AV reliability is expected to improve user interactions with automated vehicles, and to facilitate appropriate trust and improve human-automation task performance (Faltaous, Baumann, Schneegass, & Chuang, 2018, Hoff & Bashir, 2015). Based on a meta-analysis of 129 studies, Zhang et al. (2019) recommends that further efforts should be made towards ensuring that drivers are well prepared to take-over.

Aims and objectives

In our research, we will apply neurophysiological measurement to assess predictability and transparency of an automated vehicle (AV). We aim to get a better understanding of the underlying cognitive mechanisms of drivers’ predictions of the AV behaviour. We will also focus on other variables, such as age, personality, trust, and cognitive workload in the relation with the AV predictability. As a result of our research, we will derive recommendations for driver-vehicle interaction strategies that increase AV predictability.

Research description

Our first experiment will be designed as an exploratory study to determine the efficiency of communicating the state of the AV reliability to the driver, to explore the differences in drivers’ brain activity induced by the continuous peripheral visual cue, and to determine the efficiency of a peripheral visual cue as a mediator of the AV reliability.

Participants of the experiment will be involved in a simulator driving task, and simultaneous non-driving secondary task. Their eye movements will be observed, as well as the brain activity using EEG. A LED stripe in the car interior will continuously communicate the state of the AV reliability to the driver. Three light patterns representing different levels of reliability will be presented (green, yellow, and red). In some situations, the decrease in reliability will be followed by a TOR, presented in a form of red ambient light and an abstract auditory cue. Participants will have to disable the automation and take-over. We will focus on the effect of the continuous peripheral information on driver’s behaviour, performance and his or her brain activity.

Results

We expect that the results of the first experiment will prove reliability displays to be an efficient and useful tool for car automation system design. We assume the that the continuous peripheral visual cue has the potential to reduce the driver’s workload while improving the take-over time and performance. We also expect to see changes in the brain activity induced by the peripheral light, especially in the alpha- and theta-range

My publications

References and links

Borojeni, S. S., Chuang, L., Heuten, W., & Boll, S. (2016). Assisting drivers with ambient take-over requests in highly automated driving. AutomotiveUI 2016 – 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Proceedings, 237–244. https://doi.org/10.1145/3003715.3005409

Capalar, J., & Olaverri-Monreal, C. (2018). Hypovigilance in limited self-driving automation: Peripheral visual stimulus for a balanced level of automation and cognitive workload. IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, 2018March, 27–31. https://doi.org/10.1109/ITSC.2017.8317925

Cohen-Lazry, G., Borowsky, A., & Oron-Gilad, T. (2017). The effects of continuous driving-related feedback on drivers’ response to automation failures. Proceedings of the Human Factors and Ergonomics Society, 2017October, 1980–1984. https://doi.org/10.1177/1541931213601974

Collet, C., & Musicant, O. (2019). Associating vehicles automation with drivers functional state assessment systems: A challenge for road safety in the future. Frontiers in Human Neuroscience, 13(April), 1–12. https://doi.org/10.3389/fnhum.2019.00131

Faltaous, S., Baumann, M., Schneegass, S., & Chuang, L. L. (2018). Design guidelines for reliability communication in autonomous vehicles. Proceedings – 10th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2018, 258–267. https://doi.org/10.1145/3239060.3239072

Hoff, K. A., & Bashir, M. (2015). Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57(3), 407–434. https://doi.org/10.1177/0018720814547570

Jamson, A. H., Merat, N., Carsten, O. M. J., & Lai, F. C. H. (2013). Behavioural changes in drivers experiencing highly-automated vehicle control in varying traffic conditions. Transportation Research Part C: Emerging Technologies, 30, 116–125. https://doi.org/10.1016/j.trc.2013.02.008

Kunze, A., Summerskill, S. J., Marshall, R., & Filtness, A. J. (2019). Conveying uncertainties using peripheral awareness displays in the context of automated driving. Proceedings – 11th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2019, 329–341. https://doi.org/10.1145/3342197.3344537

Leibowitz, H. W., Post, R. B., Brandt, T., & Dichgans, J. (1982). Implications of Recent Developments in Dynamic Spatial Orientation and Visual Resolution for Vehicle Guidance. Tutorials on Motion Perception, 231-260. doi:10.1007/978-1-4613-3569-6_8

Löcken, A., Heuten, W., & Boll, S. (2015). Supporting lane change decisions with ambient light. Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications – AutomotiveUI ’15. doi:10.1145/2799250.2799259

Norman, D. A. (1990). The ‘problem ’ with automation: Inappropriate feedback and interaction, not ‘over-automation’. Philosophical Transactions of the Royal Society of London. B, Biological Sciences, 327(1241), 585-593. doi:10.1098/rstb.1990.0101

SAE International. (2016). Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicle (J3016) [Standard]. Warrendale, PA: SAE International.

Van Veen, T., Karjanto, J., & Terken, J. (2017). Situation awareness in automated vehicles through proximal peripheral light signals. AutomotiveUI 2017 – 9th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, Proceedings, 287–292. https://doi.org/10.1145/3122986.3122993

Wickens, C. D. (2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3(2), 159–177. https://doi.org/10.1080/14639220210123806

Yang, Y., Götze, M., Laqua, A., Caccia Dominioni, G., Kawabe, K., & Bengler, K. (2017). A method to improve driver’s situation awareness in automated driving. Proceedings of the Human Factors and Ergonomics Society Europe Chapter 2017 Annual Conference, 4959, 29–47.

Zhang, B., de Winter, J., Varotto, S., Happee, R., & Martens, M. (2019). Determinants of take-over time from automated driving: A meta-analysis of 129 studies. Transportation Research Part F: Traffic Psychology and Behaviour, 64(May), 285–307. https://doi.org/10.1016/j.trf.2019.04.020