Performance assessment of RGB-D cameras in deep learning algorithms for obstacle avoidance systems in agriculture

Run-over accidents represent one of the most common occupational hazards related to agricultural machinery and obstacle detection systems can represent a major step forward in workers’ safety. In this field of research, the use of colour (RGB) and depth cameras (D) is a possible solution to support both driving assistance technologies for tractor drivers and computer vision for autonomous vehicles. The object of this study, funded by the Italian National Institute for Insurance against Accidents at Work (INAIL), is therefore to evaluate the performance of RGB-D cameras in agricultural contexts for detection tasks at various ranges. To do so, a real-time pipeline of images has been captured by an RGB-D camera that provides both RGB and depth frames at the same pixel resolution and rate. Colour frames have been analysed through a deep-learning algorithm using a TensorFlow-lite object classification model, which has been run on a Raspberry Pi 4 coupled with a Google Coral AI accelerator. The distance of detected obstacles on the three axes have then been estimated from the corresponding depth frames to get both frontal and lateral distances this way. Tests have been run with clear illumination conditions and detection data has finally been compared to the georeferenced data of obstacles obtained with satellite navigation technology provided by Real Time Kinematics (RTK). Despite the assessed depth distances showed an absolute error of 0.84 m within 10 metres and 0f 6 m for obstacles beyond 16 metres, the lateral distance from the camera proved to be quite reliable an all ranges, with a minimum absolute error of 0.04 m and a maximum of 1.42 m at far ranges. Results showed that RGB-D cameras in agricultural environments can be particularly effective in estimating the risk of collision with obstacles in the vehicle’s trajectory.