EAL‑FUSION: AN EXPLAINABLE ADAPTIVE FUSION LAYER FOR RGB‑DEPTH‑THERMAL PERSON SEGMENTATION

Abstract

Image segmentation plays a key role in computer vision applications such as security, robotics, and autonomous systems. However, traditional RGB-based methods struggle in low-light or complex environments. This paper presents EAL-Fusion, an Explainable Adaptive Layer designed for RGB–Depth–Thermal (RDT) image segmentation. The model learns to dynamically weight modalities per pixel, producing interpretable confidence maps that highlight which sensor contributes most. Implementation was carried out in PyTorch, trained and evaluated on the TriStar Multimodal People Segmentation Dataset. The proposed model achieved an IoU of 0.4423, Dice of 0.5506, Precision of 0.6811, Recall of 0.5432, and Accuracy of 0.9759. The model performs segmentation only on human subjects (persons), not on other objects, demonstrating effective explainability and robustness under challenging conditions.