ESR 13 – Computational AV/Pedestrian Interaction Models

Popular scientific abstract

Traffic injuries and fatalities, harmful emissions, traffic jam and resulting delays in daily trips are some of the potential challenges of today’s world traffic. It’s been about three decades that a growing body of research suggests that human error is the leading cause of traffic accidents and while we should never assume accidents as single factor issue, it reveals the story to the greater extent. The Automated vehicles (AVs) and especially highly automated vehicles (HAVs) could be a potential solution for addressing these issues. However, in the coming decades, we would expect to see these robots sharing the roads with conventional cars and this makes the situation more complex. How should be the communication between different road users shaped to see a safe, convenient and transparent traffic environment? In addition, human error is not limited to drivers and at the other end of the problem, we see vulnerable road users (VRUs) and especially pedestrians. Every human experiences this role on a daily basis while travelling on his foot for various reasons and we know that human nature is unpredictable. Many reasons are behind this, but for now, let’s consider that studies show that many people have problem in estimating their abilities including walking speed and also the approaching vehicle features like speed, distance and the decision of the operator (whether it’s a human driver or a robot). Hence, we need to provide safe, reliable and pleasant communication schemes and tools regarding the above challenges. This project aims to shed light on how pedestrians interact with AVs in different scenarios such as road crossings, making predictions about for example response time patterns, time pressure effects and other factors involving decision making using computational models and based on the research outcomes, formulate recommendations for AV design, and potentially training and publicity initiatives for the human’s side.

My affiliation

Contact details to my supervisors:

Dr Gustav Markkula:

Prof Natasha Merat:

Prof Marco Dozza:

Dr Mikael Ljung Aust:


Interacting with pedestrians could be interpreted as the case of social intelligence as AVs should employ various levels of pedestrian models starting from simple visual models to more sophisticated models capable of predicting the agents’ trajectories or their interactions with each other before or during the questioned task using their social cues [1-3]. Moreover, currently, AVs are programmed in a way that called “conservative” or “defensive” which means that they would yield to other cars, for instance, in merging/diverging scenarios [3, 4] or whenever pedestrians move/jump in front of them [5] which is counter-intuitive to the philosophy of their existence. On the other hand, pedestrians showed to have lower crossing intentions when interacting with AVs rather conventional cars [6] and their behaviour and trust are dependent on AVs behaviour [7]. So, the key question here is how we can manage AV/pedestrian interactions in a way that promises a safe, efficient and transparent traffic flow? Computational models of these interactions can address some of the concerns. By defining and employing an appropriate model, we can see what would happen in crossing behaviour scenarios considering agents’ trajectories, decision time (response and reaction time) and social preference. Several studies have been conducted in this field [2] but none of them to the best of our knowledge could actually accommodate all of the factors mentioned and establish a similar connection with actual human behaviour. Thus, it may worthwhile to connect models of cognitive science to current mathematical models [8-10] as it is evident that heterogeneity of the pedestrians, drivers, vehicles, and road environment has not been clearly considered in the modelling process of the most past studies [11].

Aims and objectives

The main objective of the current project is to shed light on how pedestrians interact with AVs in different scenarios such as road crossings, making predictions about for example response time patterns, time pressure effects, and attention during deliberation process, possibly using a behavioural game theory model. In more detail we aim for:

  • Defining a set of applicable scenarios with regard to safety prominence, observed frequency of each one, and current modelling state of the art.
  • Implementing the scenarios in an experimental environment (e.g. virtual reality) to see if the proposed model can capture the road user behaviour well.
  • Model how AVs should behave in specific situations, to meet pedestrian expectations, avoiding to transgress human comfort boundaries, and take pedestrians’ intentions into account.

Research description

This study seeks to evaluate existing computational models capable of defining road users interactions on one hand and recognise the important factors in pedestrian/AV interaction scenarios on the other hand. With this method, current models are being identified regarding their accuracy and adaptability and their possibility to be revised or extended taking the important parameters into account. Moreover, scenarios in question would be considered and identified with respect to safety prominence and frequency of their occurrence according to the literature. Thus, the first phase of the study revolves around meticulous and comprehensive literature review and the second phase would be building the model using programming languages.

After identifying the potential model and set of scenarios, experimental studies will be conducted using virtual reality environments and specifically the Highly Immersive Kinematic Experimental Research (HIKER) pedestrian lab to see to what extent the primary model could explain the parameters and capture the human behaviour well. From initial results, the model would be revised/modified and if the results were not satisfactory enough, alternative models would be employed instead. This process continues until getting a favourable outcome.

Beside experimental studies, several surveys using questionnaires may be conducted to measure participants subjective evaluation and our proposed model capability. Additionally, national datasets may be used to see the performance of the model regarding actual traffic situations.


Expected results include:

  • A model capable of accommodating critical parameters of human decision making and motion path will be obtained.
  • The results of the experimental studies using the HIKER lab would be very close to the results obtained from naturalistic datasets.
  • The developed models constitute intellectual property and could be exploited by academics or industrial partners, for example in AV algorithms or in computer simulations for virtual AV testing. This will be discussed with the industrial partner, Volvo Car Corporation, in the secondment.
  • Based on the research outcomes, formulated recommendations for AV design, and potentially training and publicity initiatives for the human’s side will be discussed.

My publications

To come…

References and links

1.           Camara, F., et al., Pedestrian Models for Autonomous Driving Part I: low level models, from sensing to tracking. arXiv preprint arXiv:2002.11669, 2020.

2.           Camara, F., et al., Pedestrian Models for Autonomous Driving Part II: high level models of human behaviour. Under submission to IEEE Transactions on Intelligent Transportation Systems, 2020.

3.           Schwarting, W., et al., Social behavior for autonomous vehicles. Proceedings of the National Academy of Sciences, 2019. 116(50): p. 24972-24978.

4.           Sadigh, D., et al., Planning for cars that coordinate with people: leveraging effects on human actions for planning and active information gathering over human internal state. Autonomous Robots, 2018. 42(7): p. 1405-1426.

5.           Fox, C., et al., When should the chicken cross the road?: Game theory for autonomous vehicle-human interactions. 2018.

6.           Velasco, J.P.N., et al., Studying pedestrians’ crossing behavior when interacting with automated vehicles using virtual reality. Transportation research part F: traffic psychology and behaviour, 2019. 66: p. 1-14.

7.           Jayaraman, S., et al., Pedestrian Trust in Automated Vehicles: Role of Traffic Signal and AV Driving Behavior. 2019.

8.           Wright, J.R. and K. Leyton-Brown. Behavioral game theoretic models: a Bayesian framework for parameter analysis. in AAMAS. 2012.

9.           Golman, R., S. Bhatia, and P.B. Kane, The dual accumulator model of strategic deliberation and decision making. Psychological Review, 2019.

10.         Wright, J.R. and K. Leyton-Brown, Predicting human behavior in unrepeated, simultaneous-move games. Games and Economic Behavior, 2017. 106: p. 16-37.

11.         Amado, H., et al., Pedestrian–Vehicle Interaction at Unsignalized Crosswalks: A Systematic Review. Sustainability, 2020. 12(7): p. 2805.