Autonomous Driving RC Car Pipeline (Embedded System & AI Integration)

This project represents the integration of embedded systems with advanced artificial intelligence, specifically focusing on autonomous driving capabilities using a Raspberry Pi. The core of the system relies on a dual-AI pipeline: a custom Convolutional Neural Network (CNN) for steering angle prediction and YOLOv5 for real-time object detection.

🚀 System Overview

The pipeline operates by capturing real-time video feed from a camera module, processing the frames through two parallel AI models, and translating the predictions into physical motor control actions to navigate the environment.

Hardware Stack

• Controller: Raspberry Pi
• Actuators: DC Motors controlled via PWM
• Motor Driver: L298N / Dual H-Bridge using RPi.GPIO
• Sensor: USB Web Camera / PiCamera

🧠 AI & Software Pipeline

1. Image Acquisition & Multi-threading

First of all, for training, I manually make driving road using paper and collect image for training.ㄴ This is the raw images before preprocessing used for training driving model. ㄴ This is the images labelled manually for training YOLO.
I paid attention to maintaining a balanced image distribution to avoid driving model bias.

The system uses OpenCV to capture video frames in real-time. To maintain a high frame rate and responsiveness, the pipeline utilizes Python's threading module.

• Main Thread: Handles image preprocessing, CNN inference for steering, and motor control.
• Background Thread: Runs the YOLOv5 object detection to prevent the heavy computation from blocking the driving logic.

2. Steering Angle Prediction (Custom CNN)

The steering logic is driven by a custom PyTorch Convolutional Neural Network, inspired by the NVIDIA End-to-End Autonomous Driving model.

• Image Preprocessing:
  • Cropping: The top portion of the image is removed to focus solely on the road/lane.
  • Resizing & Grayscale: Scaled down to 200x66 pixels and converted to grayscale.
  • Filtering: Gaussian Blur is applied to reduce noise, followed by Image Thresholding (cv2.THRESH_BINARY_INV) to highlight lane features.
  • Normalization: Standardized using PyTorch's transforms.Normalize.
  Final preprocessed image is shwon above.
• Model Architecture: A multi-layer CNN consisting of 5 Convolutional layers and 4 Fully Connected (Linear) layers, with Batch Normalization and Dropout to prevent overfitting.
• Output: The network classifies the image into one of three discrete steering angles (45°, 90°, 135°), representing Left, Straight, and Right.

3. Real-Time Object Detection (YOLO)

While the CNN handles where the car should turn, a YOLOv5 model acts as the car's "eyes" for understanding its environment.

• Detection Targets: Identifies specific objects and environmental markers such as crossroad, emptyBottle, and laneBottle.
• Decision Logic:
  • Object bounding box sizes and relative positions are passed from the background thread via thread-safe queues.
  • Dynamic Intervention: If an obstacle or a crossroad is detected within a certain distance threshold, the system automatically slows down or triggers a full motor stop, overriding the CNN's driving commands to prioritize safety.

4. Motor Control Execution

Based on the combined inferences from the CNN and YOLOv5, the system calculates the final speed and steering angle. Using the RPi.GPIO library, PWM (Pulse Width Modulation) signals are dispatched to the Left and Right motors. Duty cycles are adjusted to execute physical maneuvers smoothly, completing the end-to-end autonomous driving loop.

🎯 Summary

This project successfully bridges the gap between high-level AI algorithms and low-level hardware control. By decoupling the steering prediction (CNN) from the object detection (YOLO) using multi-threading, the embedded system achieves real-time autonomous navigation, speed adaptation, and dynamic obstacle avoidance.