YOLO Object Detection + ZED Stereo Camera

Suzie Petryk, Joe Nechleba, MinYoung Chang

This project is an extension of what was done as a part of AI-DO (Artificial Intelligence Driving Olympics) by duckietown: https://www.duckietown.org/research/ai-driving-olympics Suzie Petryk and Joe Nechleba worked on training a YOLO model specific for finding ducks, and I worked on implementing the YOLO model on ZED Camera and come up with an algorithm for depth measurement of detected objects.

---

Integration with ZED Camera

I found the result of the project above very interesting, and decided to implement it for ZED Stereo Camera that is connected with JACKAL CLEARPATH. The real-time object detection of YOLO makes a great synergy with the ZED Stereo Camera, whcih outputs the real-time position of camera, depth map, and path trajectory. Implementing the feature to the system could be a great asset for Cornell Autonomous Systems Lab.

Depth Calculation

As shown in the video above, ZED generates a depth map based on the measured distance of each pixel. Since the depth map is generated in the same image format as ordinary image format, it is very easy to find out the distance between the camera and an object detected in an image. Following is the video of pre-trained yolo model ran in ZED Stereo Camera.

Running the System on JACKAL through ROS

System Flowchart