YOLOv10s based human victim detection in cluttered urban environments with a crawler type rescue robot

Rescue robots are widely utilized in search and rescue operations to enhance operations’ efficiency. To reduce operators’ load a robot could perform some functions automatically, including victims’ detection. This paper introduces a robot operating system based victim detection framework for Servosila Engineer crawler rescue robot with four cameras. The victim detection algorithm employs video stream frames from a single camera and a trained YOLOv10s neural network that detects human body parts within a picture of a cluttered urban environment. To train the YOLOv10s model, a human body dataset of 15068 images was created by combining an existing dataset with a new dataset collected with the robot’s camera. The model was trained to detect a person and his/her body parts: a head, a hand, and a foot. The study analyzed an impact of a distance between the robot and a human victim in cluttered environments on detection accuracy. The algorithm showed acceptable performance in validation experiments with three human participants under artificial lighting conditions when the robot’s camera was positioned within 50 to 200 cm distance from a cluttered area. Within this distance, an Average Precision (AP) of 0.75, 0.91, and 0.73 was achieved for the head, hand, and foot classes respectively; the AP rapidly degraded with distance. The experiments showed that hand class objects were detected more reliably compared to other objects across all three intervals. Unlike prior approaches that employed high-end hardware or multiple cameras, our system achieved a reasonable accuracy using a single camera and low-power onboard computing.