Project Background

This project is part of the Google Summer of Code (GSoC) 2025 program. The mentoring organization is The Linux Foundation, through its Automotive Grade Linux (AGL) open source project. AGL is one of The Linux Foundation’s projects, aimed at accelerating the development of a fully open software stack for connected cars.

Intelligent Camera Stream Integration and Warning System (ICSW)

The primary goal of this project is to develop a camera-based application tailored to automotive use cases. Cameras play a crucial role in the automotive perception stack, from classifying static and moving objects to recognizing traffic light status. This project ultimately aims to implement an in-vehicle warning system that issues alerts when objects enter the vehicle’s blind spot.

The planned subtasks for the GSoC 2025 contribution include:

  1. Integrating inference models (e.g., object detection, depth estimation) as PipeWire nodes

  2. Retrieving raw and processed camera streams as Flutter overlays

  3. Enabling dynamic reconfiguration of the PipeWire graph at runtime

Community Bonding Period (May 8 – June 1)

During the Community Bonding Period, I engaged with the AGL community primarily through the AGL website and the community Discord channel. The project proposal served as the foundation for initiating communication with my GSoC mentors — Jan-Simon, Walt, Joel, George, and other AGL members.

Throughout the application period, my mentors provided a clear vision, which helped me to consolidate and refine my ideas in the proposal. I used the bonding period to become more familiar with the AGL ecosystem. As I:

  1. Participated in the AGL Weekly Developer Call
  2. Set up my host development environment
  3. Built the AGL master branch and ran a demo image on QEMU

I also began exploring PipeWire to list the available camera devices on my host machine along with their supported formats:

$ wpctl status
Video
├─ Devices:
│      34. Integrated_Webcam_HD                [v4l2]
│      35. Integrated_Webcam_HD                [v4l2]
├─ Sinks:
├─ Sources:
│      36. Integrated_Webcam_HD               
├─ Filters:
└─ Streams:

$ pw-dump 36
{
"mediaType": "video",
"mediaSubtype": "raw",
"format": "YUY2",
"size": { "width": 160, "height": 120 },
"framerate": { "num": 30, "denom": 1 }
}

Next steps

To tweak the video-play.c script of the PipeWire API src/examples/video-play.c to retrieve the camera stream from my host machine