Route-structured perception for railway scene understanding using forward-looking camera data

Loading...
Thumbnail Image

Authors

Ghorbanalivakili, Mohammadjavad

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

Railway perception is not only a problem of detecting obstacles or infrastructure elements. For safe railway automation, the important question is how scene elements relate to the route that a train can occupy. An object near the railway corridor may have different safety implications depending on whether it lies outside all reachable tracks, intersects a possible rail path, or lies on the valid ego-path. Since trains cannot steer around hazards, perception systems must represent the route structure that constrains future motion. This thesis therefore studies railway scene understanding as a structured perception problem centered on ego-path interpretation.

The thesis develops this idea through a sequence of methods for forward-looking imagery. First, path perception is formulated as a triplet-based extraction problem, where left rail, right rail, and centerline information are used to generate possible ego-path proposals in multi-track scenes. This representation connects low-level rail detection to route-level interpretation by preserving which tracks form plausible paths ahead of the train. The thesis then advances this formulation through learned ego-path instance tracing, using an attraction-field representation to reduce reliance on heuristics and to support route continuity through complex layouts.

The thesis further addresses the distinction between possible paths and the valid route. In switch scenes, several paths may be visible, but only one may correspond to the route the train is set to follow. This problem is investigated through reasoning-guided segmentation with a railway-adapted vision--language model, which is prompted to identify the valid ego-path and may generate an auxiliary rationale intended to describe switch geometry. Joint semantic and reasoning-segmentation finetuning produces higher ego-path overlap than semantic-only finetuning, with larger observed gains in switch-dependent scenes. This stage moves the thesis from route proposal generation toward route validity interpretation.

Overall, this thesis contributes an integrated account of railway scene understanding in which ego-paths serve as enabling representations for downstream safety reasoning. The thesis identifies remaining challenges in occlusion, long-range ambiguity, switch interpretation, topology-aware evaluation, explanation quality, and dataset coverage.

Description

Keywords

Artificial intelligence, Transportation planning

Citation