Predicting and understanding road user behaviors through big geo-referenced egocentric video data analytics

Understanding and forecasting road user behaviors is an important problem for road safety, transportation, and autonomous driving. With the advancement in technology, rich data have been collected from the real world, such as GPS trajectories, surveillance camera data, and egocentric (i.e., first-person view) video records from helmet/vehicle-mounted cameras. Deciphering the behavior of road users to predict their future actions and what they would do from videos can generate insights on how to keep these road users safe and bring inspiration to facilitate behavioral studies. Traditional analyses are done by manually observing the video data to code them and extract useful information, which are not scalable to larger datasets due to intensive human involvement and introduce potential human bias. This thesis aims to develop deep learning-based methods to investigate the behavior of cyclists and drivers, such as how they respond to the various traffic conditions, how they react in the complex road environment, and how they interact with other road users. Specifically, we utilize well-known and reliable object detection techniques to identify key factors (e.g., moving vehicles, pedestrians, road conditions and infrastructure) in the sight of a road user and propose methods to automatically learn the relationship between the presence of such factors and the decisions made by the road user. Case studies and extensive experiments on large-scale real-world data show that our proposed solutions demonstrate their efficacy and outperform baseline methods.