Evaluation of Multi-Modal Object Detection and Fusion Strategies for Automatic Train Operation
8th International Congress on Human-Computer Interaction, Optimization and Robotic Applications, ICHORA 2026, Ankara, Türkiye, 21 - 23 Mayıs 2026, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/ichora69329.2026.11537158
- Basıldığı Şehir: Ankara
- Basıldığı Ülke: Türkiye
- Anahtar Kelimeler: automatic train operation, multimodal detection, Sensor fusion
- Galatasaray Üniversitesi Adresli: Evet
Özet
Multimodal object detection improves the performance of Automatic Train Operation. Perception of signals, tracks, transitions, and dynamic objects such as pedestrians and trains over long distances is necessary for advanced railway automation, particularly Grade of Automation 3 (GoA3) and Grade of Automation 4 (GoA4), corresponding to driverless and unattended train operation, respectively. In this study, three sensing modalities formed by the suite of a RGB, an infrared (IR), and a radar sensor, two fusion schemes such as early fusion using 4-channel Radar and RGB encoding, and late fusion through Weighted Boxes Fusion (WBF) are evaluated and tested. YOLOv8 and YOLOv11 are used as one-stage detector to analyze the impact of dense convolutions versus attention-based feature extraction in the deep learning detector architecture. The OSDaR23 dataset is used. The results illustrate that early fusion systematically improves over single-modality baselines and YOLOv11 outperforms YOLOv8. Multimodality improves detection of small and distant objects in the railway scene under low-visibility conditions, and radar provides complementary structural cues. Mean average precision (mAP) metric is improved by early fusion and multimodality of RGB and 4-channel radar using YOLOv11 by 2.1% versus the RGB single modality. By fusing radar and RGB sensor, the multimodality improves the mAP by 2.45% versus the RGB sensor stand-alone modality and 125.14% for the radar modality using the day dataset, and by 1.79% for RGB and 376.77% for radar using the night dataset. This study contributes to ATO object detection research by evaluating and testing multimodality with early and late fusion, and two one-stage detectors with different modules in their architectures.