The Role of Context in Understanding and Predicting Pedestrian Behavior in Urban Traffic Scenes

Today, one of the major challenges faced by autonomous vehicles (AVs) is the ability to drive in urban environments. Such a task requires interactions between AVs and other road users, in particular pedestrians, to resolve various traffic ambiguities. To interact with pedestrians, AVs must be able to understand the objectives of pedestrians and predict their forthcoming actions. In this dissertation, we investigate the role of context on understanding and predicting pedestrian behavior in urban traffic scenes. Towards this goal, we begin by explaining why behavior prediction is necessary for social interactions. Next, we conduct a meta-analysis of a large body of behavioral literature and identify the factors that potentially impact pedestrian behavior and how these factors are interconnected. We extend the past findings by conducting two behavioral studies of pedestrians. The first study shows that pedestrians often engage in different forms of communication, mainly implicit, with changes in their movement patterns and the frequency of communication varying depending on road structure, social factors, and scene dynamics. The second study identifies the diversity of pedestrian behavioral patterns at the time of crossing and how it is influenced by factors such as the road width, demographics, crosswalk delineation, and driver behavior. As part of the behavioral studies, we collected two novel large-scale datasets of pedestrian crossing behaviors. Using the data, we empirically evaluate various state-of-the-art and classical pedestrian detection algorithms and show how diversifying training data in terms of visual properties, such as lighting conditions and pedestrian attributes, enhance the generalizability of such algorithms. Furthermore, we propose a novel pedestrian trajectory prediction algorithm that achieves state-of-the-art performance. We show that incorporating pedestrian intention to cross helps improve reasoning about future motion trajectories. In addition, we propose a novel pedestrian crossing action prediction algorithm and illustrate that by including contextual information, such as pedestrian appearance, pedestrian pose, and velocity, we can enhance the accuracy of crossing action prediction. We also show that by combining different modalities of contextual data in a hierarchical fashion better performance can be achieved compared to alternative approaches.