Optimizing Real-Time VRU Detection Workloads for Heterogeneous Clusters and the Edge
The use case “Vulnerable Road User (VRU) Safety in Intelligent Intersections” (Use Case 1), led by VeNIT Lab (Marmara University) and BigTRI, is one of the use cases in the DECICE project, which demonstrates the framework’s ability to work in C-ITS (Cooperative Intelligent Transportation Systems) environments. Designed based on the Car2Car Communication Consortium’s (C2C-CC) definition of Vulnerable Road User Protection (VRUP) application [1], the developed system improves VRU safety in intersections using real-time image processing and Infrastructure-to-Vehicle (I2V) communications, as detailed in a previous news article [2].
The DECICE Framework introduces a transformative capability that transcends the specific use case of VRU safety, offering system-wide benefits across the broader landscape of Cooperative Intelligent Transportation Systems (C-ITS). By integrating advanced features such as AI-based scheduling with both network-awareness and constraint-awareness, real-time digital twin-based system monitoring, and automated management of cluster resources, DECICE addresses the complex demands of modern C-ITS infrastructures. Its intelligent scheduler leverages real-time insights into network conditions and device-level resource availability—continuously provided by digital twin models—to strategically allocate workloads across a heterogeneous cloud-edge continuum. This architecture, essential for large-scale, multi-service C-ITS environments, ensures high-performance operation through optimal task placement and efficient resource orchestration.
Deploying time-critical C-ITS workloads on resource-constrained edge devices presents several challenges. These include the need for trained models to function efficiently across varied hardware and operating system variations while maintaining low latency, high accuracy, and energy efficiency.
Addressing heterogeneity in the cluster
To address the heterogeneity in the cluster resulting from the diversity of the devices placed either on the edge or in the cloud, multiple container images have been built for the real-time Vulnerable Road User (VRU) detection application. These images are tailored to accommodate differences in CPU architecture, GPU model, and operating system within the multi-cluster test environment (for details about the multi-cluster test environment, please refer to [3]). Further, optimization of the models to enable effective execution on resource-constrained edge devices (NVIDIA Jetson Orin and NVIDIA Jetson Nano) has been carried out for preserving accuracy and keeping processing latency within the constraints that are mandated by ETSI standards on intelligent transportation systems [1].
Adapting the model for resource-constrained edge devices
Given that the edge computing cluster in the testbed comprises NVIDIA GPU-equipped devices, NVIDIA’s TensorRT SDK has been employed to optimize and execute the VRU detection model, which was developed in PyTorch. The following techniques were applied:
- FP16 (half-precision format): The model’s floating-point weights and activations were converted from FP32 to FP16. This reduced memory usage and improved computational speed, with minimal accuracy loss.
- INT8 quantization: Further compresses the model and recalibrates it using representative data to maintain accuracy. INT8 models typically provide up to 4× faster inference and 3–4× lower memory usage compared to full-precision counterparts [4].
- Layer fusion and kernel auto-tuning: TensorRT automatically fuses compatible layers to minimize memory access overhead and optimizes kernel selection based on the target hardware profile.
Although FP16 (half-precision) and INT8 (low-bit) optimization techniques improve speed, they may result in a decrease in accuracy. Therefore, after all transformations and simplifications, the model was re-evaluated on a test dataset to verify that its performance met the system’s minimum requirements.
Author: Berkay Yaman (BigTRI), İbrahim Tınas (VeNIT Lab – Marmara University), Mujdat Soyturk (VeNIT Lab – Marmara University)
References:
[1] Car 2 Car Communication Consortium, Guidance for Day 2 and Beyond Roadmap (2021), Available at: https://www.car-2-car.org/fileadmin/documents/General_Documents/C2C-CC_White_Paper_Day_2_and_Beyond_Roadmap.pdf, accessed on 23.05.2025.
[2] DECICE Project Website, Use Case 1: Enhancing Autonomous Driving with Device-Edge-Cloud Intelligent Collaboration Framework (2023), Available at: https://www.decice.eu/project-news/enhancing-autonomous-driving/, accessed on 23.05.2025.
[3] DECICE Project Website, VRU Safety In Intelligent Intersections: Demonstrating DECICE In A Multi-Cluster Environment (UC-1 of The Project DECICE) (2025), Available at: https://www.decice.eu/project-news/vru-safety-in-intelligent-intersections-demonstrating-decice-in-a-multi-cluster-environment/, accessed on 23.05.2025.
[4] NVIDIA Corporation, TensorRT Developer Guide: Quantization (2024), Available at: https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#quantization, accessed on 23.05.2025.
Keywords:
#Vulnerable Road User #smart city #Intelligent Intersections #Intelligent Transportation Systems #C-ITS #Edge Computing #Digital Twin #Real-time Image Processing #TensorRT #PyTorch #Heterogeneous Cloud-Edge Architecture
